[Paper Review] Transformation Networks for Target-Oriented Sentiment Classification

Xin Li, Lidong Bing+2 more|arXiv (Cornell University)|May 3, 2018

Sentiment Analysis and Opinion MiningComputer Science24 references58 citations

TL;DR

Introduces TNet, a CNN-based model with Target-Specific Transformation and context-preserving mechanisms to improve target-oriented sentiment classification, achieving new state-of-the-art on benchmark datasets.

ABSTRACT

Target-oriented sentiment classification aims at classifying sentiment polarities over individual opinion targets in a sentence. RNN with attention seems a good fit for the characteristics of this task, and indeed it achieves the state-of-the-art performance. After re-examining the drawbacks of attention mechanism and the obstacles that block CNN to perform well in this classification task, we propose a new model to overcome these issues. Instead of attention, our model employs a CNN layer to extract salient features from the transformed word representations originated from a bi-directional RNN layer. Between the two layers, we propose a component to generate target-specific representations of words in the sentence, meanwhile incorporate a mechanism for preserving the original contextual information from the RNN layer. Experiments show that our model achieves a new state-of-the-art performance on a few benchmarks.

Motivation & Objective

• Motivate improved target-oriented sentiment classification by addressing attention-induced noise and CNN limitations.
• Propose a novel Target-Specific Transformation (TST) module to tailor word representations to each target.
• Incorporate a context-preserving mechanism to maintain contextual information across deep transformation layers.
• Employ a proximity-based CNN feature extractor with position-aware inputs to highlight target-related sentiment words.
• Demonstrate state-of-the-art performance on multiple benchmark datasets (LAPTOP, REST, TWITTER).

Proposed method

• Encode context with BiLSTM to obtain contextualized word representations.
• Generate target-specific word representations via a dynamic Target-Specific Transformation (TST) that conditions target words on individual context words.
• Incorporate a context-preserving mechanism (Lossless Forwarding or Adaptive Scaling) to retain contextual information across CPT layers.
• Apply a proximity-based CNN with position relevance to focus on opinion words near the target, followed by max-pooling and softmax classifier.

Experimental results

Research questions

• RQ1 Can replacing attention with a target-specific transformation improve target-oriented sentiment classification?
• RQ2 Does a context-preserving, deep transformation architecture better capture target-context interactions than prior CNN or RNN approaches?
• RQ3 How does a proximity-aware CNN influence performance when targeting sentiment for specific targets?
• RQ4 What is the impact of the number of CPT layers on classification accuracy and stability?

Key findings

• TNet variants (LFa and AS) achieve the best performance across LAPTOP, REST, and TWITTER datasets.
• Target-Specific Transformation (TST) with context-preserving mechanisms significantly outperforms ablated variants and several strong baselines.
• Proximity-based position weighting improves the CNN’s ability to identify correct target-related sentiment phrases.
• Optimal CPT depth is 2 layers; deeper architectures degrade performance due to increased training difficulty.
• Replacing CPT with attention (LSTM-ATT-CNN) generally underperforms TNet on these tasks, especially for noisy data like TWITTER.
• TNet-LF and TNet-AS deliver consistent gains with statistically significant improvements over competing methods.