[Paper Review] Entity-Relation Extraction as Multi-Turn Question Answering

Xiaoya Li, Fan Yin+6 more|arXiv (Cornell University)|May 14, 2019

Topic Modeling52 references48 citations

TL;DR

The paper casts entity-relation extraction as a multi-turn question answering problem, using QA-style templates and a BERT-backed MRC model to jointly extract entities and relations, achieving new SOTA on ACE04, ACE05, and CoNLL04 and introducing RESUME for hierarchical reasoning.

ABSTRACT

In this paper, we propose a new paradigm for the task of entity-relation extraction. We cast the task as a multi-turn question answering problem, i.e., the extraction of entities and relations is transformed to the task of identifying answer spans from the context. This multi-turn QA formalization comes with several key advantages: firstly, the question query encodes important information for the entity/relation class we want to identify; secondly, QA provides a natural way of jointly modeling entity and relation; and thirdly, it allows us to exploit the well developed machine reading comprehension (MRC) models. Experiments on the ACE and the CoNLL04 corpora demonstrate that the proposed paradigm significantly outperforms previous best models. We are able to obtain the state-of-the-art results on all of the ACE04, ACE05 and CoNLL04 datasets, increasing the SOTA results on the three datasets to 49.4 (+1.0), 60.2 (+0.6) and 68.9 (+2.1), respectively. Additionally, we construct a newly developed dataset RESUME in Chinese, which requires multi-step reasoning to construct entity dependencies, as opposed to the single-step dependency extraction in the triplet exaction in previous datasets. The proposed multi-turn QA model also achieves the best performance on the RESUME dataset.

Motivation & Objective

• Motivate a unified, question-answering formulation for entity-relation extraction to capture hierarchical dependencies.
• Propose a multi-turn QA pipeline with head-entity and tail-entity/relation stages.
• Leverage MRC models (BERT) with BMEO tagging to extract multiple answers per context.
• Show state-of-the-art results on ACE04, ACE05, and CoNLL04 datasets and introduce RESUME for Chinese, multi-turn reasoning.

Proposed method

• Formulate entity and relation extraction as sequential QA turns guided by templates.
• Use two-stage extraction: head-entity extraction via entity-specific questions, followed by chained relation/tail-entity questions.
• Generate questions from natural language or pseudo-questions; fill slots with previously extracted entities.
• Adopt BERT-based MRC with BMEO tagging to predict answer spans across turns.
• Optionally apply reinforcement learning (REINFORCE) to optimize turn-by-turn extraction rewards.
• Train with a combined loss L = (1-λ)L_head-entity + λ L_tail-entity+relation; share parameters across stages.

Experimental results

Research questions

• RQ1 Can a multi-turn QA paradigm effectively model hierarchical dependencies among entities and relations in complex structures?
• RQ2 Does incorporating informative question templates improve extraction of distant or multi-hop relations compared to traditional triplet approaches?
• RQ3 What benefits do reinforcement learning and curriculum strategies offer in multi-turn QA-based entity-relation extraction?
• RQ4 How does the approach perform on standard benchmarks (ACE04, ACE05, CoNLL04) and on a new dataset requiring deeper multi-turn reasoning (RESUME)?

Key findings

• Achieves state-of-the-art results on ACE04 (Entity F1 up by 1.0), ACE05 (Entity F1 up by 1.2), and CoNLL04 (Relation F1 up by 1.1).
• Outperforms prior joint and pipelined models by utilizing a multi-turn QA framework that captures hierarchical tag dependencies.
• Natural language question templates outperform pseudo-questions across datasets.
• Reinforcement learning provides additional performance gains, particularly on more turns (e.g., RESUME).
• RESUME dataset demonstrates the model’s ability to handle multi-step reasoning with four-turn extraction for Chinese biographies.