[Paper Review] Relational Graph Attention Network for Aspect-based Sentiment Analysis

Kai Wang, Weizhou Shen+3 more|arXiv (Cornell University)|Apr 26, 2020

Sentiment Analysis and Opinion Mining31 references48 citations

TL;DR

The paper introduces R-GAT, a relational graph attention network built on an aspect-oriented dependency tree to improve ABSA. It shows state-of-the-art results, especially when combined with BERT.

ABSTRACT

Aspect-based sentiment analysis aims to determine the sentiment polarity towards a specific aspect in online reviews. Most recent efforts adopt attention-based neural network models to implicitly connect aspects with opinion words. However, due to the complexity of language and the existence of multiple aspects in a single sentence, these models often confuse the connections. In this paper, we address this problem by means of effective encoding of syntax information. Firstly, we define a unified aspect-oriented dependency tree structure rooted at a target aspect by reshaping and pruning an ordinary dependency parse tree. Then, we propose a relational graph attention network (R-GAT) to encode the new tree structure for sentiment prediction. Extensive experiments are conducted on the SemEval 2014 and Twitter datasets, and the experimental results confirm that the connections between aspects and opinion words can be better established with our approach, and the performance of the graph attention network (GAT) is significantly improved as a consequence.

Motivation & Objective

• Address misalignment between aspects and opinion words in ABSA due to complex syntax and multiple aspects per sentence.
• Propose an aspect-oriented dependency tree rooted at a target aspect to focus on relevant syntactic relations.
• Develop the Relational Graph Attention Network (R-GAT) to encode labeled dependency edges for ABSA.
• Evaluate R-GAT on SemEval 2014 Restaurant/Laptop datasets and Twitter to demonstrate improved connections between aspects and opinions and overall performance.

Proposed method

• Construct an aspect-oriented dependency tree by reshaping the ordinary dependency parse tree to root at the target aspect and prune to keep direct edges to the aspect.
• Introduce a Relational Graph Attention Network (R-GAT) that extends GAT with relation-aware heads to incorporate edge labels from dependency relations.
• Use BiLSTM to encode node representations, and a second BiLSTM to encode aspect words as the root.
• Compute node representations through multi-head attentional and relational heads with relation embeddings and gates (equations 1–7 in paper).
• Aggregate root representation and classify sentiment with a softmax layer (equations 8–9).
• Optionally combine R-GAT with BERT by feeding BERT-derived word representations and fine-tuning on ABSA task.

Experimental results

Research questions

• RQ1 How can we redesign syntactic input to ABSA to better connect target aspects with their opinion words?
• RQ2 Does an aspect-oriented dependency structure improve ABSA performance over standard dependency trees?
• RQ3 Can a relation-aware graph attention mechanism better exploit syntactic edge labels for aspect-level sentiment classification?
• RQ4 What is the impact of integrating R-GAT with pre-trained models like BERT on ABSA benchmarks?

Key findings

• R-GAT outperforms many baselines on ABSA benchmarks, and R-GAT+BERT achieves state-of-the-art results.
• The aspect-oriented tree with n:con edges improves robustness against parsing errors and long-distance dependencies.
• Relational heads enable the model to modulate information flow according to dependency relations, improving aspect–opinion word connections.
• Ablation shows the new tree structure and n:con edges contribute significantly to performance gains across datasets.
• Biaffine parsing yields better ABSA performance than Stanford parsing when used with R-GAT.
• Error analysis highlights remaining challenges such as neutral reviews with modifiers, deep language understanding needs, and double negation.