[Paper Review] Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

Suncong Zheng, Feng Wang+4 more|arXiv (Cornell University)|Jun 7, 2017

Topic Modeling27 references106 citations

TL;DR

The paper introduces a tagging scheme to convert joint entity-relation extraction into a sequence tagging task and evaluates end-to-end Bi-LSTM/LSTM models with a biased decoding objective, achieving state-of-the-art results on the NYT distant supervision dataset.

ABSTRACT

Joint extraction of entities and relations is an important task in information extraction. To tackle this problem, we firstly propose a novel tagging scheme that can convert the joint extraction task to a tagging problem. Then, based on our tagging scheme, we study different end-to-end models to extract entities and their relations directly, without identifying entities and relations separately. We conduct experiments on a public dataset produced by distant supervision method and the experimental results show that the tagging based methods are better than most of the existing pipelined and joint learning methods. What's more, the end-to-end model proposed in this paper, achieves the best results on the public dataset.

Motivation & Objective

• Motivate joint extraction of entities and relations and reduce feature engineering by transforming the task into tagging.
• Propose a novel tagging scheme that encodes entity boundaries and relation roles within tags.
• Develop end-to-end Bi-LSTM-based models with a biased decoding objective to jointly extract entity pairs and relations.

Proposed method

• Define a tagging scheme with tags of the form B/I/E/S-Relation-Role and O for irrelevant words.
• Transform triplet extraction into sequence tagging using the predefined relation set and roles.
• Implement an end-to-end model: Bi-LSTM encoder plus LSTM decoder with a biased loss to emphasize relational tags.
• Train with RMSprop on a distant-supervised NYT dataset and evaluate using precision, recall, F1 for triplets.
• Compare with pipelined, joint-learning, and other end-to-end tagging baselines.
• Bias parameter alpha is tuned to balance precision and recall.

Experimental results

Research questions

• RQ1 Can a tagging-based approach effectively convert joint entity and relation extraction into a sequence tagging problem?
• RQ2 Do end-to-end Bi-LSTM models with a biased decoding objective improve triplet extraction over existing methods on NYT-distant-supervision data?
• RQ3 How does the proposed tagging scheme handle the association between two entities in a relation and what is the impact of bias on performance?

Key findings

• The tagging-based end-to-end models outperform most pipelined and joint-learning baselines on the NYT dataset.
• LSTM-LSTM-Bias achieves the best F1 score among all tested methods, surpassing CoType by about 3 percentage points.
• Jointly extracting models generally outperform pipelined approaches, and tagging-based methods are competitive or superior to many joint methods.
• Using a biased loss increases the linkage between related entities and reduces single (unpaired) entities compared to baselines.
• The LSTM decoding with bias improves balance between precision and recall, yielding strongest F1 when alpha is around 10.