[Paper Review] Deeper Task-Specificity Improves Joint Entity and Relation Extraction

Phil Crone|arXiv (Cornell University)|Feb 15, 2020

Topic Modeling24 references4 citations

TL;DR

This paper proposes a deeper task-specificity approach in multi-task learning for joint named entity recognition (NER) and relation extraction (RE) by introducing additional task-specific bidirectional RNNs for both NER and RE. It achieves state-of-the-art performance on the ADE dataset and competitive results on CoNLL04 with 10x fewer parameters than prior SOTA, demonstrating that deeper task-specificity significantly improves performance and that shared/task-specific layer balance should be tuned per dataset.

ABSTRACT

Multi-task learning (MTL) is an effective method for learning related tasks, but designing MTL models necessitates deciding which and how many parameters should be task-specific, as opposed to shared between tasks. We investigate this issue for the problem of jointly learning named entity recognition (NER) and relation extraction (RE) and propose a novel neural architecture that allows for deeper task-specificity than does prior work. In particular, we introduce additional task-specific bidirectional RNN layers for both the NER and RE tasks and tune the number of shared and task-specific layers separately for different datasets. We achieve state-of-the-art (SOTA) results for both tasks on the ADE dataset; on the CoNLL04 dataset, we achieve SOTA results on the NER task and competitive results on the RE task while using an order of magnitude fewer trainable parameters than the current SOTA architecture. An ablation study confirms the importance of the additional task-specific layers for achieving these results. Our work suggests that previous solutions to joint NER and RE undervalue task-specificity and demonstrates the importance of correctly balancing the number of shared and task-specific parameters for MTL approaches in general.

Motivation & Objective

• To address the underutilization of task-specific parameters in existing multi-task learning models for joint NER and RE.
• To investigate whether increasing task-specificity beyond shallow architectures improves performance on joint NER and RE tasks.
• To demonstrate that the optimal balance of shared and task-specific layers varies across datasets and should be tuned per domain.
• To reduce model complexity while maintaining or improving performance through deeper task-specific design.
• To provide empirical evidence that task-specific layers are crucial, especially for more complex tasks like relation extraction.

Proposed method

• Introduces additional task-specific bidirectional RNN (BiRNN) layers for both NER and RE, enabling deeper task-specificity beyond prior work.
• Treats the number of shared and task-specific BiRNN layers as separate hyperparameters, tuned independently per dataset.
• Uses hard parameter sharing: shared layers process input first, followed by task-specific BiRNNs for NER and RE.
• Employs a BIO/BILOU labeling scheme for NER and predicts relations between entity pairs in a joint decoding framework.
• Applies the model to two datasets—ADE and CoNLL04—using identical training and evaluation protocols with random weight initialization across trials.
• Performs ablation studies by removing NER-specific, RE-specific, or all task-specific BiRNN layers while keeping total parameter count constant.

Experimental results

Research questions

• RQ1 Does increasing task-specificity in multi-task learning architectures lead to improved performance in joint NER and RE?
• RQ2 How does the performance of joint NER and RE models vary with different configurations of shared and task-specific layers across datasets?
• RQ3 Can deeper task-specificity reduce the number of trainable parameters while maintaining or improving SOTA performance?
• RQ4 Is the benefit of task-specific layers more pronounced for one task (NER or RE) than the other, and why?
• RQ5 Can the optimal balance of shared and task-specific layers be determined empirically per dataset, rather than using a fixed architecture?

Key findings

• The proposed model achieves state-of-the-art F1 scores on both NER (89.78 macro) and RE (72.63 macro) on the ADE dataset.
• On the CoNLL04 dataset, the model achieves SOTA performance on NER (89.78 macro F1) and near-SOTA on RE (71.08 micro F1), outperforming prior models.
• The model uses approximately 10 times fewer trainable parameters than the current SOTA architecture while matching or exceeding performance.
• Ablation studies show that removing RE-specific BiRNN layers causes a significant drop in RE F1 (to 55.47 micro), indicating strong dependency on task-specific capacity for RE.
• NER performance is less sensitive to removal of NER-specific layers, suggesting RE is more complex and benefits more from task-specific modeling.
• The model with no NER-specific layers but with RE-specific and shared layers outperforms Nguyen and Verspoor’s (2019) model, highlighting the benefit of deeper shared and task-specific architecture.