[Paper Review] An Embarrassingly Simple Model for Dialogue Relation Extraction

Fuzhao Xue, Aixin Sun+3 more|arXiv (Cornell University)|Dec 27, 2020

Topic Modeling4 citations

TL;DR

This paper proposes SimpleRE, a lightweight yet effective model for dialogue relation extraction that uses a novel BERT Relation Token Sequence (BRS) with multiple [CLS] tokens to capture interrelations among multiple entity pairs in a dialogue. A Relation Refinement Gate (RRG) adaptively refines relation-specific representations, achieving state-of-the-art F1 scores on DialogRE with significantly faster training than prior BERT-based methods.

ABSTRACT

Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue. In this paper, we propose a simple yet effective model named SimpleRE for the RE task. SimpleRE captures the interrelations among multiple relations in a dialogue through a novel input format named BERT Relation Token Sequence (BRS). In BRS, multiple [CLS] tokens are used to capture possible relations between different pairs of entities mentioned in the dialogue. A Relation Refinement Gate (RRG) is then designed to extract relation-specific semantic representation in an adaptive manner. Experiments on the DialogRE dataset show that SimpleRE achieves the best performance, with much shorter training time. Further, SimpleRE outperforms all direct baselines on sentence-level RE without using external resources.

Motivation & Objective

• To address the challenge of modeling interrelations among multiple relations in dialogue texts for improved relation extraction.
• To design a simple yet effective architecture that leverages BERT's strength without complex structures like graph networks.
• To reduce training time while maintaining or improving performance over existing BERT-based models.
• To demonstrate generalization by adapting the model to sentence-level relation extraction tasks.

Proposed method

• Introduces BERT Relation Token Sequence (BRS), a modified input format with multiple [CLS] tokens placed before each entity pair to encode relation-specific representations.
• Uses the first [CLS] token (h₀) to represent the full dialogue context, while subsequent [CLS] tokens (hᵣᵢ) encode relation-specific semantics for each entity pair.
• Applies a Relation Refinement Gate (RRG) that concatenates h₀ and hᵣᵢ, processes them via a single-layer feed-forward network with sigmoid activation, and computes a confidence score to gate relation prediction.
• Employs a linear classifier on the gated representation for relation classification, with softmax used in sentence-level RE.
• Designs ablation variants (BRS-v2, BRS-v3) to evaluate the importance of [CLS] token placement and the RRG module.
• Adapts the model to sentence-level RE by simplifying BRS to include only one [CLS] per sentence and removing RRG due to single-relation setting.

Experimental results

Research questions

• RQ1 Can a simple input reformatting of BERT inputs (BRS) effectively model interrelations among multiple relations in dialogue?
• RQ2 Does the proposed Relation Refinement Gate (RRG) improve relation representation learning in a way that boosts performance?
• RQ3 How does the performance of SimpleRE compare to strong baselines like BERTs and GDPNet on the DialogRE dataset?
• RQ4 Can the SimpleRE framework be effectively adapted to sentence-level relation extraction without external knowledge?
• RQ5 What is the contribution of each component (BRS, RRG, h₀, hᵣ) to the final performance, as shown through ablation studies?

Key findings

• SimpleRE achieves the highest F1 score on the DialogRE dataset among all BERT-based models, outperforming BERTs and GDPNet by a large margin.
• The model trains at least 5 times faster than BERTs and GDPNet, demonstrating significant efficiency gains.
• Ablation studies show that removing the second [CLS] token (used for relation representation) reduces F1 by 1.1%, confirming its importance.
• Performance drops to 70.0 F1 when only the relation representation hᵣ is used, and to 70.6 when only the context representation h₀ is used, proving both components are essential.
• On the TACRED and TACRED-Revisit datasets, SimpleRE achieves 71.7 and 80.7 F1, respectively, outperforming all direct baselines including KnowBERT, which uses external knowledge.
• The BRS-v2 and BRS-v3 variants, which reposition [CLS] tokens using [SEP], show significant performance drops (62.8 and 63.5 F1), indicating that [CLS] position sensitivity is critical.