[Paper Review] An Improved Baseline for Sentence-level Relation Extraction

Wenxuan Zhou, Muhao Chen|arXiv (Cornell University)|Feb 2, 2021

Topic Modeling23 references49 citations

TL;DR

The paper proposes typed entity markers to improve sentence-level relation extraction and achieves state-of-the-art F1 scores on TACRED, TACREV, and Re-TACRED, notably 91.1% on Re-TACRED using RoBERTa-LARGE with punct-based typed markers.

ABSTRACT

Sentence-level relation extraction (RE) aims at identifying the relationship between two entities in a sentence. Many efforts have been devoted to this problem, while the best performing methods are still far from perfect. In this paper, we revisit two problems that affect the performance of existing RE models, namely entity representation and noisy or ill-defined labels. Our improved RE baseline, incorporated with entity representations with typed markers, achieves an F1 of 74.6% on TACRED, significantly outperforms previous SOTA methods. Furthermore, the presented new baseline achieves an F1 of 91.1% on the refined Re-TACRED dataset, demonstrating that the pretrained language models (PLMs) achieve high performance on this task. We release our code to the community for future research.

Motivation & Objective

• Highlight the impact of entity representation on sentence-level RE performance.
• Investigate how typed entity markers compare to other entity representation techniques.
• Evaluate the improved baseline on TACRED, TACREV, and Re-TACRED datasets.
• Assess robustness to noisy labels and unseen entities.

Proposed method

• Formalize sentence-level RE with a classifier that uses subject and object entity embeddings from a PLM."
• Represent entities with various techniques, including typed entity markers that include NER types in the input."
• Incorporate the best-performing entity representation into a RoBERTa-LARGE backbone and fine-tune end-to-end."
• Compare performance across TACRED, TACREV, and Re-TACRED datasets."],
• research_questions([

Experimental results

Research questions

• RQ1 What is the effect of different entity representation techniques on sentence-level RE performance?
• RQ2 Can typed entity markers improve RE accuracy over untyped markers or masks?
• RQ3 How does the improved baseline perform on TACRED, TACREV, and Re-TACRED compared to prior SOTA methods?
• RQ4 Does the approach maintain robustness when labels are noisy or entities are unseen?

Key findings

• Typed entity markers (punct) with RoBERTa-LARGE achieve 74.6% F1 on TACRED, 83.2% on TACREV, and 91.1% on Re-TACRED.
• Typed entity markers consistently outperform entity masks and non-typed markers across encoders.
• The improved baseline with typed entity markers sets new SOTA on TACRED, TACREV, and Re-TACRED (RoBERTa-LARGE + typed marker (punct)).
• On the clean TACRED test set, gains from typed markers are smaller than on TACRED/TACREV, indicating label noise effects in TACRED-derived datasets.
• The study demonstrates that PLMs can achieve high RE performance when entity type information is effectively integrated into input representations.
• The authors provide evidence that entity names can aid generalization to unseen entities when using typed markers.