[Paper Review] Exploring Conditional Text Generation for Aspect-Based Sentiment Analysis

Siva Uday Sampreeth Chebolu, Franck Dernoncourt+2 more|arXiv (Cornell University)|Oct 5, 2021

Sentiment Analysis and Opinion Mining4 citations

TL;DR

This paper proposes a novel conditional text generation framework for aspect-based sentiment analysis (ABSA), transforming the task into generating auxiliary statements conditioned on targets, aspects, and polarities. By fine-tuning T5 and BART on benchmark datasets, the method achieves new state-of-the-art results, particularly improving implicit target detection and eliminating target conflict issues inherent in prior sequence labeling and sentence-pair classification approaches.

ABSTRACT

Aspect-based sentiment analysis (ABSA) is an NLP task that entails processing user-generated reviews to determine (i) the target being evaluated, (ii) the aspect category to which it belongs, and (iii) the sentiment expressed towards the target and aspect pair. In this article, we propose transforming ABSA into an abstract summary-like conditional text generation task that uses targets, aspects, and polarities to generate auxiliary statements. To demonstrate the efficacy of our task formulation and a proposed system, we fine-tune a pre-trained model for conditional text generation tasks to get new state-of-the-art results on a few restaurant domains and urban neighborhoods domain benchmark datasets.

Motivation & Objective

• To address the limitations of existing ABSA methods that struggle with implicit targets and target conflicts in sentiment triple extraction.
• To reformulate ABSA as a conditional text generation task to unify multiple sub-tasks (target detection, aspect categorization, polarity prediction) under a single sequence-to-sequence framework.
• To leverage pre-trained models like T5 and BART to generate expressive, context-aware auxiliary statements that encode aspect-sentiment relationships.
• To demonstrate that joint generation outperforms task-specific fine-tuning and improves performance across diverse ABSA sub-tasks.
• To validate that the improvement stems from the generative formulation rather than just model scale or pre-training data size.

Proposed method

• Reformulate ABSA as a conditional text generation task where the model generates a natural language statement given input context, target, aspect category, and sentiment polarity.
• Construct training data using pseudo-phrases and pseudo-sentences that encode the (target, aspect, sentiment) triple, enabling end-to-end generation of opinion statements.
• Fine-tune T5 and BART models on four public ABSA benchmark datasets, including restaurant and urban neighborhoods domains, using a sequence-to-sequence objective.
• Use a multi-task learning framework that jointly optimizes for target detection, aspect categorization, and sentiment prediction through conditional generation.
• Apply CRF or softmax decoding on token-level representations for baseline comparisons, while the proposed method relies solely on autoregressive generation.
• Evaluate the T5-encoder variant to isolate the contribution of the generative architecture from that of model size or pre-training data scale.

Experimental results

Research questions

• RQ1 Can aspect-based sentiment analysis be effectively reformulated as a conditional text generation task to improve joint extraction of targets, aspects, and sentiments?
• RQ2 Does the proposed generative framework outperform existing sequence labeling and sentence-pair classification methods in terms of F1 scores across multiple ABSA sub-tasks?
• RQ3 To what extent does the conditional generation approach enhance the detection of implicit targets compared to prior methods?
• RQ4 Is the performance gain due to the generative modeling approach itself, or merely due to the use of larger pre-trained models?
• RQ5 How does the joint generation framework handle target conflicts that arise in sentence-pair classification approaches?

Key findings

• The proposed conditional text generation framework achieves new state-of-the-art results on four public ABSA benchmark datasets, including those from the restaurant and urban neighborhoods domains.
• The method significantly improves performance on implicit target detection, which is often missed by prior classification-based approaches.
• Joint training via conditional generation outperforms individual fine-tuning of sub-tasks, demonstrating the benefit of multi-task modeling in this formulation.
• The T5-encoder variant, pre-trained on a larger corpus than BERT, achieves superior results, confirming that the improvement stems from the generative architecture rather than just pre-training data size.
• The framework successfully resolves target conflict issues present in Wan et al. (2020)’s sentence-pair classification approach, where redundant or conflicting opinions were discarded.
• The ablation study confirms that the generative approach is more expressive and effective than traditional sequence labeling or classification methods for ABSA.