[Paper Review] Entity-Specific Sentiment Classification of Yahoo News Comments

Prakhar Biyani, Cornelia Caragea+1 more|arXiv (Cornell University)|Jun 11, 2015

Sentiment Analysis and Opinion Mining21 references3 citations

TL;DR

This paper proposes a two-stage approach for entity-specific sentiment classification in Yahoo News comments, using context-aware entity extraction and novel non-lexical features (e.g., entity type, grammatical role) to improve sentiment detection. The method outperforms strong baselines, achieving an F-1 score of 0.700 on sentiment polarity classification, with IsPerson and IsSubjObj features proving most informative.

ABSTRACT

Sentiment classification is widely used for product reviews and in online social media such as forums, Twitter, and blogs. However, the problem of classifying the sentiment of user comments on news sites has not been addressed yet. News sites cover a wide range of domains including politics, sports, technology, and entertainment, in contrast to other online social sites such as forums and review sites, which are specific to a particular domain. A user associated with a news site is likely to post comments on diverse topics (e.g., politics, smartphones, and sports) or diverse entities (e.g., Obama, iPhone, or Google). Classifying the sentiment of users tied to various entities may help obtain a holistic view of their personality, which could be useful in applications such as online advertising, content personalization, and political campaign planning. In this paper, we formulate the problem of entity-specific sentiment classification of comments posted on news articles in Yahoo News and propose novel features that are specific to news comments. Experimental results show that our models outperform state-of-the-art baselines.

Motivation & Objective

• To address the gap in sentiment analysis for user comments on multi-domain news platforms like Yahoo News, where sentiment is tied to specific entities rather than fixed targets.
• To overcome challenges such as irrelevant entities and implicit sentiment (e.g., irony, rhetorical questions) in news comment sentiment classification.
• To design and evaluate novel non-lexical features—such as entity type (person/non-person) and grammatical role (subject/object)—for improved sentiment detection.
• To develop a context extraction algorithm that identifies relevant entities in comments, filtering out those not tied to sentiment expression.
• To demonstrate that comment-specific features (e.g., sentiment strength, question marks) further enhance polarity classification performance.

Proposed method

• The method uses a two-stage classification pipeline: first identifying sentiment-relevant entities, then classifying sentiment polarity toward them.
• A context extraction algorithm identifies the entity and its surrounding context in comments, improving feature relevance and reducing noise from irrelevant entities.
• Non-lexical features such as IsPerson (whether the entity is a person) and IsSubjObj (whether the entity is a subject or object) are introduced to capture syntactic and semantic roles.
• Comment-specific features—such as NumNeg (number of negative words), PosVsNeg (ratio of positive to negative words), and IsQuesMark (presence of question marks)—are used to model sentiment intensity and rhetorical cues.
• The model combines bag-of-words (BoW) features with part-of-speech (POS) tags and sentiment lexicon scores (SentiStrength) to enhance polarity classification.
• Feature importance is evaluated using Information Gain (IG), which ranks IsPerson and IsSubjObj as the most informative features, outperforming lexical-based features like HasClues and SentiStrength.

Experimental results

Research questions

• RQ1 Can context-aware entity extraction improve the identification of sentiment-relevant entities in news comments?
• RQ2 How effective are non-lexical features—such as entity type and grammatical role—in distinguishing sentiment-relevant from irrelevant entities?
• RQ3 To what extent do comment-specific features (e.g., sentiment strength, question marks) enhance polarity classification beyond traditional lexical and POS features?
• RQ4 How do implicit sentiment expressions (e.g., irony, rhetorical questions) affect sentiment classification, and can the model detect them effectively?
• RQ5 What is the relative contribution of different features to sentiment classification performance, and which are most informative?

Key findings

• The proposed model achieves an F-1 score of 0.700 on sentiment polarity classification, significantly outperforming all baselines, including SentiStrength (feature-based) with F-1 = 0.674.
• The removal of the IsPerson feature reduces the F-1 score to 0.574, indicating it is the most informative feature, followed by IsSubjObj (F-1 = 0.636).
• The feature ranking via Information Gain confirms that IsPerson > IsSubjObj > SentiNeg > HasClues > SentiPos, showing non-lexical features are more informative than lexical ones.
• Adding comment-specific features (CSF) improves the F-1 score from 0.687 to 0.700, demonstrating their value in capturing rhetorical and sentiment intensity cues.
• The model outperforms NaiveContextExtraction, which achieves only F-1 = 0.491, showing the importance of accurate context extraction.
• The model’s performance is robust to implicit sentiment, as evidenced by high performance on examples involving irony and rhetorical questions, where traditional models fail.