[Paper Review] A Sequence-to-Set Network for Nested Named Entity Recognition

Zeqi Tan, Yongliang Shen+3 more|arXiv (Cornell University)|May 19, 2021

Topic Modeling30 references5 citations

TL;DR

This paper proposes a sequence-to-set network for nested named entity recognition (NER), treating entity prediction as an unordered set generation task to overcome limitations of sequence-to-sequence and span-based methods. By using learnable entity queries and a non-autoregressive decoder with self-attention, the model captures inter-entity dependencies and achieves state-of-the-art performance on ACE 2004, ACE 2005, and KBP 2017, outperforming prior methods by up to 2.99% in F1 score.

ABSTRACT

Named entity recognition (NER) is a widely studied task in natural language processing. Recently, a growing number of studies have focused on the nested NER. The span-based methods, considering the entity recognition as a span classification task, can deal with nested entities naturally. But they suffer from the huge search space and the lack of interactions between entities. To address these issues, we propose a novel sequence-to-set neural network for nested NER. Instead of specifying candidate spans in advance, we provide a fixed set of learnable vectors to learn the patterns of the valuable spans. We utilize a non-autoregressive decoder to predict the final set of entities in one pass, in which we are able to capture dependencies between entities. Compared with the sequence-to-sequence method, our model is more suitable for such unordered recognition task as it is insensitive to the label order. In addition, we utilize the loss function based on bipartite matching to compute the overall training loss. Experimental results show that our proposed model achieves state-of-the-art on three nested NER corpora: ACE 2004, ACE 2005 and KBP 2017. The code is available at https://github.com/zqtan1024/sequence-to-set.

Motivation & Objective

• To address the limitations of sequence-to-sequence models in nested NER, which are sensitive to label order due to autoregressive generation.
• To overcome the high search space and lack of interaction between spans in span-based methods by replacing explicit span enumeration with learnable entity queries.
• To model nested named entities as an unordered set prediction task, enabling end-to-end, one-pass entity set generation.
• To improve performance by capturing dependencies between entities through self-attention among entity queries.
• To develop a training objective based on bipartite matching that is more suitable for unordered set prediction than cross-entropy loss.

Proposed method

• The model uses a sequence encoder (BERT or BiLSTM) to encode input text into contextual representations.
• A fixed set of learnable entity queries is introduced to represent potential entities, which are updated during training to learn patterns of valuable spans.
• A non-autoregressive decoder uses cross-attention between the sequence encoding and entity queries to predict the final entity set in a single forward pass.
• Self-attention among entity queries enables direct modeling of dependencies between predicted entities.
• A bipartite matching-based loss function is used to compute the overall training loss, aligning predicted and gold entity sets without requiring label order.
• The model is trained end-to-end with the bipartite matching loss, which is more suitable for unordered set prediction than standard cross-entropy loss.

Experimental results

Research questions

• RQ1 Can treating nested NER as a sequence-to-set task improve performance by avoiding label-order sensitivity inherent in sequence-to-sequence models?
• RQ2 Can learnable entity queries effectively replace explicit span enumeration and reduce search space while maintaining or improving accuracy?
• RQ3 Does a non-autoregressive decoder with self-attention among entity queries better capture inter-entity dependencies than autoregressive or span-based methods?
• RQ4 Is bipartite matching loss more effective than cross-entropy loss for training unordered set prediction in nested NER?
• RQ5 Can the proposed model achieve state-of-the-art performance across multiple nested NER benchmarks without relying on complex pre/post-processing or error-prone span generation pipelines?

Key findings

• The proposed model achieves state-of-the-art F1 scores on three major nested NER benchmarks: 87.26 on ACE 2004, 87.05 on ACE 2005, and 85.75 on KBP 2017.
• The model outperforms strong baselines by 0.56% in F1 on ACE 2004, 1.65% on ACE 2005, and 2.99% on KBP 2017.
• Freezing the entity queries leads to a 0.40% and 0.81% drop in F1 on ACE 2004 and ACE 2005, respectively, showing that they learn meaningful span patterns.
• Replacing cross-entropy loss with bipartite matching loss improves F1 by 2.67% on ACE 2004 and 1.11% on ACE 2005, confirming its effectiveness for unordered set prediction.
• The ablation study confirms that both the learnable entity queries and the bipartite matching loss are critical components for the model's performance.