undefined | Better HN

0 pointsnl3y ago0 comments

> Sentiment analysis trained on the standard datasets is one place where performance is barely better than old-school linear classifiers

Well yeah. But why would you do that?

Do what eveyrone does: Train on large scale a language corpus (or use a pre-trained model) then finetune for sentiment analysis.

> I strongly advice against using something fine-tuned solely on sentiment datasets

Did you mean trained on sentiment datasets? I agree with that.

Otherwise, well [1] is a decent overview of the field. I think Document Vectors using Cosine Similarity[2] at 17 is the highest rated that isn't a NN trained on large corpus and fine-tune on sentiment task. Even that uses document vectors that are trained on a large language corpus.

[1] https://paperswithcode.com/sota/sentiment-analysis-on-imdb

[2] https://paperswithcode.com/paper/the-document-vectors-using-...

0 comments

Vetch3y ago

No, I meant finetuned. I also meant finetuned when I said trained. Experience with applying finetuned sentiment classifiers on real world data found gain vs cost of running to not be worth it. They remain nearly as brittle as cheaper classifiers and have a habit of gloming too much unto certain adjectives. They are also prone to overfitting on finetuned data's domain. Transformers trained not specifically on sentiment but on general domains like question answering or entailment are just leagues better for sentiment tasks.

nlOP3y ago

Well when you put the sentiment head on a pretrained language model you kind of have to train that head a bit on the sentiment task right?

But if the rest of your model is frozen the head will never see actual words, just contextual vectors from the LM.

It feels like we are in strong agreement but using slightly different terms or something

j / k navigate · click thread line to collapse