[Paper Review] GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis

Huaishao Luo, Lei Ji+3 more|arXiv (Cornell University)|Sep 22, 2020

Sentiment Analysis and Opinion Mining45 references4 citations

TL;DR

This paper proposes GRACE, a cascaded labeling framework for aspect-based sentiment analysis that enhances sentiment polarity prediction by leveraging aspect term labels through multi-head attention, while addressing class imbalance via a gradient harmonized loss. The model achieves state-of-the-art performance on multiple benchmark datasets by improving interaction between aspect terms and balancing training signals.

ABSTRACT

In this paper, we focus on the imbalance issue, which is rarely studied in aspect term extraction and aspect sentiment classification when regarding them as sequence labeling tasks. Besides, previous works usually ignore the interaction between aspect terms when labeling polarities. We propose a GRadient hArmonized and CascadEd labeling model (GRACE) to solve these problems. Specifically, a cascaded labeling module is developed to enhance the interchange between aspect terms and improve the attention of sentiment tokens when labeling sentiment polarities. The polarities sequence is designed to depend on the generated aspect terms labels. To alleviate the imbalance issue, we extend the gradient harmonized mechanism used in object detection to the aspect-based sentiment analysis by adjusting the weight of each label dynamically. The proposed GRACE adopts a post-pretraining BERT as its backbone. Experimental results demonstrate that the proposed model achieves consistency improvement on multiple benchmark datasets and generates state-of-the-art results.

Motivation & Objective

• To address the under-explored class imbalance issue in aspect term extraction and sentiment classification when modeled as sequence labeling tasks.
• To improve sentiment polarity labeling by modeling interactions between aspect terms through a cascaded architecture.
• To enhance attention to sentiment-relevant tokens by conditioning polarity labeling on predicted aspect term boundaries.
• To extend the gradient harmonized loss from object detection to sequence labeling in aspect-based sentiment analysis.
• To achieve state-of-the-art performance on aspect term-polarity co-extraction through end-to-end training with post-pretraining and domain fine-tuning.

Proposed method

• GRACE employs a cascaded labeling architecture where aspect term labels from the ATE branch are used as input to the ASC branch to enable interaction between aspect terms.
• The ASC branch uses a stacked multi-head attention mechanism to attend to aspect terms and improve sentiment prediction contextually.
• A gradient harmonized loss (GHL) is applied to dynamically adjust the loss weight of each label, reducing the dominance of frequent 'O' labels and improving training on rare aspect terms.
• The model uses a post-pretrained BERT backbone for contextual representation, with shared lower layers across both ATE and ASC branches.
• Virtual adversarial training and domain-specific post-training are applied to further improve generalization and performance on target datasets.
• The joint sequence labeling framework treats both ATE and ASC as parallel sequence labeling tasks, avoiding the pipeline error propagation of cascaded models.

Experimental results

Research questions

• RQ1 How can aspect term interactions be effectively modeled to improve sentiment polarity classification in aspect-based sentiment analysis?
• RQ2 To what extent does class imbalance in sequence labeling tasks for ABSA affect model performance, and how can it be mitigated?
• RQ3 Can gradient harmonization, originally used in object detection, be effectively adapted to sequence labeling tasks in ABSA to improve learning on rare labels?
• RQ4 Does a cascaded labeling strategy that conditions sentiment labeling on aspect term predictions lead to better performance than independent or joint labeling?
• RQ5 How do virtual adversarial training and domain-specific post-training contribute to the robustness and generalization of the model?

Key findings

• GRACE achieves state-of-the-art F1 scores of 70.71% on the SemEval-14 laptop dataset and 78.07% on the restaurant dataset, outperforming the BASE model (68.35% and 76.76%) without cascaded labeling.
• The ablation study confirms that the cascaded labeling strategy improves performance by enabling inter-aspect term interaction and enhancing attention to sentiment-relevant tokens.
• The gradient harmonized loss significantly reduces the impact of dominant 'O' labels, as evidenced by improved performance on rare aspect terms and more balanced gradient distributions.
• Case studies show that GRACE correctly identifies sentiment even in low-contrast cases, demonstrating the effectiveness of both cascaded labeling and loss reweighting.
• The model with virtual adversarial training and domain post-training further improves performance, indicating the benefit of robust training and domain adaptation.
• The results confirm that the proposed framework effectively addresses both label imbalance and aspect term interaction, leading to consistent improvements across multiple benchmarks.