[Paper Review] Target-Dependent Sentiment Classification with Long Short Term Memory.

Duyu Tang, Bing Qin+2 more|arXiv (Cornell University)|Dec 3, 2015

Topic ModelingComputer Science20 references105 citations

TL;DR

This paper proposes two target-dependent Long Short-Term Memory (LSTM) models that explicitly incorporate target word information into sequence modeling to improve sentiment classification. By dynamically capturing the semantic relationship between a target and its context, the models achieve state-of-the-art performance on a Twitter benchmark without relying on syntactic parsers or external sentiment lexicons.

ABSTRACT

Target-dependent sentiment classification remains a challenge: modeling the semantic relatedness of a target with its context words in a sentence. Different context words have different influences on determining the sentiment polarity of a sentence towards the target. Therefore, it is desirable to integrate the connections between target word and context words when building a learning system. In this paper, we develop two target dependent long short-term memory (LSTM) models, where target information is automatically taken into account. We evaluate our methods on a benchmark dataset from Twitter. Empirical results show that modeling sentence representation with standard LSTM does not perform well. Incorporating target information into LSTM can significantly boost the classification accuracy. The target-dependent LSTM models achieve state-of-the-art performances without using syntactic parser or external sentiment lexicons.

Motivation & Objective

• Address the challenge of target-dependent sentiment classification, where sentiment polarity depends on the relationship between a target and its surrounding context.
• Overcome the limitation of standard LSTMs, which fail to model the semantic relevance between a target and its context words effectively.
• Develop a neural architecture that automatically incorporates target information during sequence encoding to enhance sentiment prediction.
• Achieve state-of-the-art performance on a benchmark Twitter dataset without using syntactic parsers or sentiment lexicons.
• Demonstrate that modeling target-context interactions improves classification accuracy over standard LSTM approaches.

Proposed method

• Design two variants of target-dependent LSTM models that integrate the target word representation into the input gate and cell state of the LSTM unit.
• Use a learned target representation that is dynamically combined with context word embeddings during sequence processing.
• Apply a bidirectional LSTM to capture long-range dependencies in both forward and backward directions relative to the target.
• Train the model end-to-end using cross-entropy loss for multi-class sentiment classification.
• Employ word embeddings as input features, with target-specific attention mechanisms to emphasize relevant context words.
• Optimize the model using stochastic gradient descent with backpropagation through time to update parameters.

Experimental results

Research questions

• RQ1 Can standard LSTM models effectively capture the semantic relationship between a target and its context for sentiment classification?
• RQ2 How does explicitly integrating target information into the LSTM architecture affect classification performance?
• RQ3 To what extent can target-dependent LSTMs achieve state-of-the-art results without relying on external resources like sentiment lexicons or syntactic parsers?
• RQ4 What is the impact of modeling target-context interactions on sentiment polarity prediction accuracy?
• RQ5 How do the proposed models compare to existing approaches in terms of performance and architectural simplicity?

Key findings

• Standard LSTM models perform poorly on target-dependent sentiment classification due to insufficient modeling of target-context semantic relationships.
• Incorporating target information into the LSTM architecture significantly improves classification accuracy on the Twitter benchmark dataset.
• The proposed target-dependent LSTM models achieve state-of-the-art performance without using syntactic parsers or external sentiment lexicons.
• The models demonstrate that explicit modeling of target-context interactions leads to more accurate sentiment prediction than generic sequence modeling.
• The performance gain is attributed to the model's ability to dynamically weigh context words based on their relevance to the target.
• The results validate the effectiveness of end-to-end learning with target-aware LSTM units in capturing nuanced sentiment expressions.