[Paper Review] Discontinuous Named Entity Recognition as Maximal Clique Discovery

Yucheng Wang, Bowen Yu+4 more|arXiv (Cornell University)|Jun 1, 2021

Topic Modeling32 references4 citations

TL;DR

This paper proposes Mac, a novel named entity recognition model that formulates discontinuous NER as maximal clique discovery in a segment graph, where nodes represent candidate spans and edges link segments of the same entity. By jointly learning segment extraction and edge prediction via a 2D grid tagging scheme in a single stage, Mac eliminates exposure bias and achieves state-of-the-art performance with up to 3.5 F1-point improvement and 5x inference speedup over prior methods.

ABSTRACT

Named entity recognition (NER) remains challenging when entity mentions can be discontinuous. Existing methods break the recognition process into several sequential steps. In training, they predict conditioned on the golden intermediate results, while at inference relying on the model output of the previous steps, which introduces exposure bias. To solve this problem, we first construct a segment graph for each sentence, in which each node denotes a segment (a continuous entity on its own, or a part of discontinuous entities), and an edge links two nodes that belong to the same entity. The nodes and edges can be generated respectively in one stage with a grid tagging scheme and learned jointly using a novel architecture named Mac. Then discontinuous NER can be reformulated as a non-parametric process of discovering maximal cliques in the graph and concatenating the spans in each clique. Experiments on three benchmarks show that our method outperforms the state-of-the-art (SOTA) results, with up to 3.5 percentage points improvement on F1, and achieves 5x speedup over the SOTA model.

Motivation & Objective

• To address exposure bias in existing discontinuous NER models that rely on sequential, multi-stage prediction pipelines.
• To improve recognition of overlapping and discontinuous entity mentions, especially in clinical and complex text.
• To eliminate the discrepancy between training (using gold intermediate results) and inference (using model-predicted outputs) in sequential models.
• To develop a method that jointly learns segment boundaries and entity connectivity in a single stage, avoiding error propagation.
• To achieve both higher accuracy and faster inference speed compared to state-of-the-art transition-based and combination-based models.

Proposed method

• Construct a segment graph per sentence, where each node is a candidate span (continuous or part of a discontinuous entity), and edges connect segments belonging to the same entity.
• Use a 2D grid tagging scheme to independently predict segment boundaries (segment extraction) and inter-segment connections (edge prediction) in one stage.
• Model segment extraction as a labeling task assigning tags to token pairs to identify span boundaries, enabling detection of overlapping and discontinuous spans.
• Model edge prediction as alignment of boundary tokens from the same entity, using a learned attention mechanism over the grid to capture long-range dependencies.
• Reformulate the final NER task as non-parametric maximal clique discovery in the constructed graph, where each clique corresponds to a complete entity mention.
• Use a greedy or exact maximal clique detection algorithm to extract all valid entities from the graph, ensuring no overlap and maximal coverage.

Experimental results

Research questions

• RQ1 Can discontinuous NER be effectively reformulated as a graph-based maximal clique discovery problem?
• RQ2 Does joint learning of segment boundaries and entity connections in a single stage eliminate exposure bias in sequential models?
• RQ3 Can a 2D grid tagging framework jointly model overlapping and discontinuous spans more effectively than sequential pipelines?
• RQ4 Does the proposed Mac model achieve superior performance and efficiency compared to state-of-the-art models on discontinuous NER benchmarks?
• RQ5 How robust is Mac to varying interval and span lengths in discontinuous mentions?

Key findings

• Mac achieves state-of-the-art performance on three discontinuous NER benchmarks, with up to 3.5 percentage points absolute improvement in F1 score over the previous SOTA.
• On the CADEC dataset, Mac improves F1 by 3.5 pts compared to the previous best result, demonstrating strong effectiveness on clinical text.
• Mac achieves a 5x speedup in inference, processing 193.3 sentences per second on a Tesla V100 GPU, compared to 36.3 sentences per second for the SOTA model Trans E.
• The model maintains high performance across different span and interval lengths, with consistent superiority in most settings, especially for longer spans and larger intervals.
• Mac shows robustness even when interval length is 1 and span length is 3, though performance dips slightly in these rare cases, suggesting a challenging edge case.
• The ablation study confirms that joint learning via the 2D grid tagging scheme is critical, as it eliminates exposure bias and enables end-to-end optimization.