8

I am researching Sentiment Analysis over social media, particularly classifying online texts such as blog posts as positive, negative or neutral.

Most of the approaches I have found for sentiment analysis are supervised (they need labeled data to train a classifier). However, I have also found a couple of papers that do it using joint topic-sentiment models (unsupervised) like this one.

According to the results in the topic model papers, the main advantage of unsupervised approaches based on topic models is that they do no need any labeled data (apart from prior "general" sentiment information, i.e. a dictionary of positive/negative words). However, they do not reach the accuracy of a supervised approach (2% less of accuracy).

Are there any other advantages/disadvantages for using topic-sentiment models for sentiment classification instead of supervised approaches?

Thanks.

kanzen_master
  • 1,235
  • 3
  • 15
  • 22
  • 2
    Seems interesting to me that you can do classification this way without supervised learning. It seems intuitive that there would be more accurate supervised approaches because the supervised approaches can take advantage of the added information in the labels. – Michael R. Chernick Jul 17 '12 at 10:39
  • Thanks! That is true, supervised approaches employ very rich labeled data, compared to simple sentiment priors of unsupervised... – kanzen_master Jul 18 '12 at 10:45

2 Answers2

5

One disadvantage of an unsupervised method like LDA is it will generally take considerably longer to train compared to supervised methods. I'm also confused about the 2% increase you mention, based on table 2 it looks like an 8% difference between the best supervised approach they compared against and their best unsupervised model.

While I generally like the idea of "how far can you push unsupervised learning", sentiment seems like a poor fit in pracitce. I say this because sentiment analysis is one of the domains where it's easiest (cost, effort) to get labeled data due to the massive amount of reviews and review like content available on the internet. If your ultimate goal is to classify accurately, even the unsupervised paper you linked seems to suggest you will be better off spending your time scraping this data, as opposed to spending your time building dictionaries of positive negative words and incorporating priors.

alto
  • 3,538
  • 17
  • 20
  • Thanks! I share your opinion. I feel that topic-sentiment word cluster examples shown in that and other papers are far from reliable. By the way, what other goals apart from classifying accurately there can be for Sentiment Analysis ? – kanzen_master Jul 18 '12 at 10:44
  • 1
    I meant your goal personally. Perhaps you were trying to expand upon the research or something along those lines. – alto Jul 18 '12 at 11:23
  • The advantage of going unsupervised is that you are not getting any biases which are incorporated when your training set is sampled and labeled. It also doesn't get affected by the label noise and finally, the problem of overfitting on a held-out set or your training set is not as enlarged as in supervised methods. Eg: The LDA method would have no problems with identifying the string "F**k" as a bad sentiment word because the surrounding distribution of words are similar to that of the actual swear word whereas a supervised method would not fare as much. – madCode Aug 25 '12 at 23:51
  • 2
    I disagree. Vanilla LDA without incorporating priors (as they do in the paper I was commenting on) or using labeled data after the fact would absolutely have a problem identifying any kind of sentiment reliably. Why? Because there's no reason to assume the topics found by LDA have any relation to sentiment. In regards to your example, LDA represents documents as a bag-of-words and is therefore subject to the same limits on ability to use context as any other method (supervised or unsupervised) using this representation. – alto Aug 26 '12 at 15:26
0

As an additional note, by using unsupervised model, possible to have more domain specific result, especially if you are interested in a outlier domain. In that case it is not easy to find a good dataset for supervised learning thus you might need to use one of the unsupervised method. Or an semi-unsupervied method like, creating a lexicon list for your specific domain using one of the algorithm (there are some) than using that lexicon list you might classify your text with a supervised method.

erogol
  • 1,427
  • 1
  • 15
  • 26