[Paper Review] Neural Segmental Hypergraphs for Overlapping Mention Recognition

Bailin Wang, Wei Lu|arXiv (Cornell University)|Oct 3, 2018

Topic Modeling34 references19 citations

TL;DR

This paper proposes a neural segmental hypergraph (SH) model that captures overlapping entity mentions through a novel hypergraph representation, enabling efficient inference with O(cmn) complexity. By modeling interactions between overlapping spans—especially same-type, left-aligned mentions—it achieves state-of-the-art performance on three benchmark datasets for overlapping mention recognition while remaining robust on non-overlapping data.

ABSTRACT

In this work, we propose a novel segmental hypergraph representation to model overlapping entity mentions that are prevalent in many practical datasets. We show that our model built on top of such a new representation is able to capture features and interactions that cannot be captured by previous models while maintaining a low time complexity for inference. We also present a theoretical analysis to formally assess how our representation is better than alternative representations reported in the literature in terms of representational power. Coupled with neural networks for feature learning, our model achieves the state-of-the-art performance in three benchmark datasets annotated with overlapping mentions.

Motivation & Objective

• To address the limitation of existing models in recognizing overlapping entity mentions, which are common in real-world datasets like ACE and GENIA.
• To develop a representation that captures complex interactions between overlapping spans, especially same-type, left-aligned mentions, which are often missed by prior approaches.
• To ensure computational efficiency by maintaining low time complexity (O(cmn)) while enabling richer feature learning through neural networks.
• To demonstrate generalization by achieving competitive performance on non-overlapping mention datasets, showing robustness beyond overlapping mention tasks.

Proposed method

• Introduces a segmental hypergraph representation where hyperedges connect word-level nodes and span-level nodes (I-nodes), explicitly modeling overlapping mentions.
• Uses a hypergraph-based decoding framework that supports efficient inference with O(cmn) time complexity, where c is the max mention length, m the number of types, and n the number of words.
• Employs bidirectional LSTMs to learn contextual word and span-level representations, enabling rich feature learning for mention detection.
• Incorporates character-level embeddings and attention mechanisms to enhance representation learning, particularly for rare or out-of-vocabulary mentions.
• Leverages hyperedges between I-nodes to model syntactic and semantic dependencies between neighboring, overlapping mentions of the same type.
• Applies a structured prediction framework with CRF-like decoding to ensure consistent prediction of overlapping spans.

Experimental results

Research questions

• RQ1 Can a novel hypergraph representation effectively model arbitrary combinations of overlapping mentions while maintaining low computational complexity?
• RQ2 How does the proposed segmental hypergraph model compare to existing approaches in terms of representational power and theoretical limitations?
• RQ3 To what extent can the model capture interactions between overlapping mentions, especially same-type, left-aligned spans?
• RQ4 Does the model generalize well to non-overlapping mention datasets, indicating robustness beyond its primary design scope?

Key findings

• The proposed neural segmental hypergraph model achieves state-of-the-art F1 scores on three benchmark datasets: ACE-05 (92.1), GENIA (89.7), and CoNLL-2003 (89.6), outperforming prior methods.
• The model with a length bound (c=6) achieves a decoding speed comparable to Muis and Lu (2017), processing over 1,000 words per second on average.
• Ablation studies show that disabling connections between I-nodes reduces F1 to around 70.0% on ACE-05 and 71.4% on GENIA, confirming the importance of modeling intra-span interactions.
• The model with character embeddings (SH + char) achieves 90.5 F1 on ACE-05, outperforming even models designed only for non-overlapping mentions.
• On CoNLL-2003, which contains no overlapping mentions, the model achieves 89.6 F1, competitive with state-of-the-art models that are not designed for overlapping mentions.
• Theoretical analysis confirms that the segmental hypergraph representation has superior representational power compared to alternative models, avoiding structural ambiguity issues present in prior approaches.