[Paper Review] Towards Effective Multi-Task Interaction for Entity-Relation Extraction: A Unified Framework with Selection Recurrent Network

Xu An Wang, Ao Liu+2 more|arXiv (Cornell University)|Feb 15, 2022

Topic Modeling4 citations

TL;DR

This paper proposes a unified cascade framework that reformulates entity-relation extraction as three span-extraction tasks—entity, subject, and subject-oriented object extraction—to eliminate the form gap between NER and RE. It introduces a selection recurrent network (SRN) for implicit multi-task interaction and two fusion strategies (early and late) for sequential information flow, achieving state-of-the-art performance on ACE05 and SciERC with significant mutual benefits between NER and RE.

ABSTRACT

Entity-relation extraction aims to jointly solve named entity recognition (NER) and relation extraction (RE). Recent approaches use either one-way sequential information propagation in a pipeline manner or two-way implicit interaction with a shared encoder. However, they still suffer from poor information interaction due to the gap between the different task forms of NER and RE, raising a controversial question whether RE is really beneficial to NER. Motivated by this, we propose a novel and unified cascade framework that combines the advantages of both sequential information propagation and implicit interaction. Meanwhile, it eliminates the gap between the two tasks by reformulating entity-relation extraction as unified span-extraction tasks. Specifically, we propose a selection recurrent network as a shared encoder to encode task-specific independent and shared representations and design two sequential information propagation strategies to realize the sequential information flow between NER and RE. Extensive experiments demonstrate that our approaches can achieve state-of-the-art results on two common benchmarks, ACE05 and SciERC, and effectively model the multi-task interaction, which realizes significant mutual benefits of NER and RE.

Motivation & Objective

• Address the limited mutual benefit between NER and relation extraction (RE) in existing joint models due to task form mismatch and conflicting representations.
• Overcome the limitations of pipeline methods that ignore cross-task interaction and fail on out-of-triple entities.
• Mitigate performance degradation in NER caused by implicit interaction in shared encoders, especially for out-of-triple entities.
• Design a unified framework that enables both implicit interaction and explicit sequential information flow between NER and RE.
• Achieve state-of-the-art results by effectively modeling multi-task interaction while preserving task-specific and shared representations.

Proposed method

• Reformulate entity-relation extraction as a multi-turn span-extraction task: entity extraction, subject extraction, and subject-oriented object extraction.
• Use a BERT-based embedder and a shared selection recurrent network (SRN) encoder to generate token representations with task-specific independent and shared features.
• Design two sequential information propagation strategies: early fusion (inserting entity and subject markers into input) and late fusion (reusing encoded representations and injecting prior info as embeddings).
• Implement a selection mechanism in SRN to dynamically attend to task-specific and shared components within shared representations, reducing interference.
• Train the model end-to-end with joint optimization over all three tasks using a unified label space.
• Use a late fusion strategy to improve inference speed by reusing encoded representations, reducing computational cost compared to early fusion.

Experimental results

Research questions

• RQ1 Can a unified framework that treats NER and RE as span-extraction tasks reduce the task-form gap and improve multi-task interaction?
• RQ2 Does explicit sequential information flow from NER to RE enhance performance, especially for out-of-triple entities?
• RQ3 Can a selection recurrent network effectively separate task-specific and shared representations to minimize interference in joint learning?
• RQ4 How does the choice between early and late fusion strategies affect performance and inference efficiency?
• RQ5 To what extent does joint training of NER and RE lead to mutual improvement, particularly on challenging out-of-triple entities?

Key findings

• The proposed framework achieves state-of-the-art performance on ACE05 and SciERC, outperforming previous SOTA models in both NER and RE F1 scores.
• The early fusion strategy achieves higher performance (69.2 F1 on NER, 38.7 on RE) but is slower (9.04 sentences/sec), while the late fusion strategy offers a 32.7× speedup (295.04 sentences/sec) with only a minor drop in performance.
• On out-of-triple entities, the model with early fusion outperforms PFN by 3.8% on SciERC and 1.0% on ACE05, demonstrating superior handling of entities not involved in relations.
• Ablation studies confirm that removing the one-way information flow from NER to RE reduces NER F1 by 3.0% and RE F1 by 3.1%, proving the effectiveness of sequential propagation.
• Joint training significantly improves NER performance compared to independent training, showing that RE provides meaningful supervision to NER.
• Using all gold entities as candidate subjects (instead of only ground-truth subjects) improves RE F1 by 5%, reducing exposure bias in the object module.