[Paper Review] Description Based Text Classification with Reinforcement Learning

Duo Chai, Wei Wu+3 more|arXiv (Cornell University)|Feb 8, 2020

Topic ModelingComputer Science62 references31 citations

TL;DR

The paper reframes text classification as a QA-style task by attaching a class description to each category, and automatically learns descriptions via reinforcement learning to improve performance across single-label, multi-label, and multi-aspect sentiment tasks.

ABSTRACT

The task of text classification is usually divided into two stages: {\it text feature extraction} and {\it classification}. In this standard formalization categories are merely represented as indexes in the label vocabulary, and the model lacks for explicit instructions on what to classify. Inspired by the current trend of formalizing NLP problems as question answering tasks, we propose a new framework for text classification, in which each category label is associated with a category description. Descriptions are generated by hand-crafted templates or using abstractive/extractive models from reinforcement learning. The concatenation of the description and the text is fed to the classifier to decide whether or not the current label should be assigned to the text. The proposed strategy forces the model to attend to the most salient texts with respect to the label, which can be regarded as a hard version of attention, leading to better performances. We observe significant performance boosts over strong baselines on a wide range of text classification tasks including single-label classification, multi-label classification and multi-aspect sentiment analysis.

Motivation & Objective

• Address the limitation of traditional label-only classification that lacks explicit guidance on what to classify.
• Introduce a QA-style formulation where each class is paired with a description.
• Automatically generate class descriptions via reinforcement learning (extractive or abstractive).
• Show that description-based classification attends to salient text and improves performance across diverse tasks.

Proposed method

• Formulate classification as associating each class with a natural language description q_y.
• Use BERT as backbone and either an N-binary classifier or an N-class classifier depending on setup.
• Extract or generate class descriptions; train description generation and classification end-to-end via reinforcement learning.
• For extraction, select a text span as the description using a policy over token spans and optimize with REINFORCE using the classification reward.
• For abstractive generation, use a Seq2Seq generator with REGS to assign rewards to partial generations and stabilize training.
• Three description construction strategies: template (Tem), extractive (Ext), and abstractive (Abs).

Experimental results

Research questions

• RQ1 Can attaching descriptive labels to categories improve text classification performance compared to conventional label-embedding approaches?
• RQ2 How do automatically generated class descriptions (via templates, extractive spans, or abstractive generation) impact performance across single-label, multi-label, and multi-aspect sentiment tasks?
• RQ3 What are the effects of description quality and generation strategy on model convergence and data efficiency?

Key findings

• Description-based methods consistently outperform the BERT baseline on tested tasks.
• Template descriptions improve over BERT, with further gains from extractive and abstractive strategies.
• Extractive and abstractive strategies yield stronger gains on multi-label and multi-aspect tasks, with substantial improvements on certain datasets (e.g., BeerAdvocate and TripAdvisor).
• Longer texts benefit more from description-based guidance, acting as a hard attention mechanism.
• RL-based description generation can be initialized with templates and still converge, with extractive methods converging faster than abstractive ones.
• Using descriptions reduces error rates relative to baselines across AGNews, 20news, DBPedia, Yahoo, YelpP, IMDB, Reuters, AAPD, BeerAdvocate, and TripAdvisor datasets.