[Paper Review] Language-Independent Sentiment Analysis Using Subjectivity and Positional Information

Veselin Raychev, Preslav Nakov|arXiv (Cornell University)|Nov 28, 2019

Sentiment Analysis and Opinion Mining8 references21 citations

TL;DR

This paper proposes a language-independent sentiment analysis method that enhances sentiment polarity classification by combining positional weighting and subjectivity estimation for words and bigrams. Using a multinomial Naïve Bayes classifier, it achieves 89.85% accuracy on the standard movie review dataset—rivaling the best language-independent results without external linguistic resources or parsers.

ABSTRACT

We describe a novel language-independent approach to the task of determining the polarity, positive or negative, of the author's opinion on a specific topic in natural language text. In particular, weights are assigned to attributes, individual words or word bi-grams, based on their position and on their likelihood of being subjective. The subjectivity of each attribute is estimated in a two-step process, where first the probability of being subjective is calculated for each sentence containing the attribute, and then these probabilities are used to alter the attribute's weights for polarity classification. The evaluation results on a standard dataset of movie reviews shows 89.85% classification accuracy, which rivals the best previously published results for this dataset for systems that use no additional linguistic information nor external resources.

Motivation & Objective

• To develop a language-independent sentiment polarity classification system that does not rely on external linguistic resources such as parsers, POS taggers, or stemmers.
• To improve sentiment classification accuracy by incorporating positional information and subjectivity likelihoods of individual words and bigrams.
• To evaluate the effectiveness of subjectivity estimation as a means to weight attributes without requiring language-specific linguistic tools.
• To demonstrate that language-independent features, when enhanced with subjectivity and position, can rival or surpass results from more complex systems using linguistic analysis.
• To explore the potential of using subjectivity scores to refine feature weighting and sentence-level prioritization in sentiment classification.

Proposed method

• The method uses a multinomial Naïve Bayes classifier with attribute weights derived from word position and estimated subjectivity probability.
• Subjectivity probability is computed in two steps: first, sentence-level subjectivity likelihood is estimated using a separate subjectivity dataset; second, these probabilities are used to adjust attribute weights.
• Attribute weights are position-dependent, with higher weights assigned to words or bigrams appearing in more central or prominent positions within sentences.
• The approach applies a 0+q weighting scheme, where q controls the influence of position, with q=1 yielding optimal performance.
• The system filters out objective sentences using the subjectivity dataset and re-sorts remaining sentences by subjectivity likelihood to improve feature relevance.
• The method does not use part-of-speech tagging, dependency parsing, or stemming, preserving language independence.

Experimental results

Research questions

• RQ1 Can sentiment classification accuracy be improved using only position and subjectivity information without linguistic analysis?
• RQ2 How does incorporating subjectivity likelihood into attribute weighting affect sentiment polarity classification performance?
• RQ3 Does sorting sentences by subjectivity score lead to better classification results compared to using raw sentence order?
• RQ4 Can a language-independent model achieve competitive accuracy without external linguistic resources like parsers or POS taggers?
• RQ5 What is the impact of combining unigrams and bigrams with position- and subjectivity-based feature weighting on classification performance?

Key findings

• The proposed method achieves 89.85% accuracy on the standard movie review dataset, which is statistically significant and rivals the best previously published language-independent results.
• The baseline method using only unigrams and bigrams with position-dependent weights achieves 87.81% accuracy, a statistically significant improvement over the 85.59% baseline without subjectivity or position weighting.
• Using the subjectivity dataset to sort sentences by subjectivity likelihood yields 89.38% accuracy, but this improvement is not statistically significant compared to the baseline.
• The highest accuracy of 89.85% is achieved when combining unigrams, bigrams, and position-dependent weights with subjectivity-based attribute weighting.
• The method outperforms the best previously reported language-independent results and even surpasses some systems using linguistic features, such as the 88.1% accuracy reported by Matsumoto et al. using SVM with unigrams and bigrams.
• The 95% Wilson confidence interval for the best result (89.85%) is [88.45%, 91.10%], confirming its statistical significance over prior language-independent approaches.