[Paper Review] Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering

Rui Zhang, Cícero Nogueira dos Santos+3 more|arXiv (Cornell University)|May 13, 2018

Topic Modeling3 citations

TL;DR

This paper proposes a neural coreference resolution model that jointly optimizes mention detection and mention clustering using biaffine attention for antecedent scoring. By incorporating a mention detection loss and a biaffine attention mechanism, the model achieves state-of-the-art performance with 69.2% F1 on the CoNLL-2012 English test set using a 5-model ensemble.

ABSTRACT

Coreference resolution aims to identify in a text all mentions that refer to the same real-world entity. The state-of-the-art end-to-end neural coreference model considers all text spans in a document as potential mentions and learns to link an antecedent for each possible mention. In this paper, we propose to improve the end-to-end coreference resolution system by (1) using a biaffine attention model to get antecedent scores for each possible mention, and (2) jointly optimizing the mention detection accuracy and the mention clustering log-likelihood given the mention cluster labels. Our model achieves the state-of-the-art performance on the CoNLL-2012 Shared Task English test set.

Motivation & Objective

• To improve end-to-end neural coreference resolution by jointly optimizing mention detection and mention clustering.
• To reduce reliance on hand-crafted features and syntactic parsers by using deep neural networks with minimal feature engineering.
• To address the challenge of high proportions of non-anaphoric (singleton) mentions by explicitly modeling anaphoricity through a mention detection loss.
• To enhance antecedent scoring accuracy by replacing feed-forward networks with biaffine attention mechanisms.
• To improve generalization to unseen mentions by leveraging mention detection loss during training.

Proposed method

• Uses bidirectional LSTMs with attention to encode spans and produce contextual representations.
• Applies a feed-forward network to predict mention scores, with a loss function that explicitly optimizes mention detection accuracy.
• Employs a biaffine attention mechanism to compute antecedent scores between candidate spans, modeling the probability of coreference directly.
• Jointly optimizes two objectives: mention detection loss and mention clustering log-likelihood, using gold cluster labels.
• Considers all spans up to 10 words as potential mentions, eliminating the need for a separate mention detector.
• Uses pre-trained word embeddings and CNN-based character embeddings to enrich word representations.

Experimental results

Research questions

• RQ1 Can joint optimization of mention detection and clustering improve coreference resolution performance compared to separate training?
• RQ2 Does replacing feed-forward networks with biaffine attention in antecedent scoring lead to better coreference prediction?
• RQ3 Can a mention detection loss improve the model's ability to detect mentions not present in the training data?
• RQ4 How does the model perform on long spans, which are often under-detected in prior systems?
• RQ5 To what extent does the joint training strategy reduce cascading errors from pipeline systems?

Key findings

• The model achieves a new state-of-the-art F1 score of 69.2% on the CoNLL-2012 English test set using a 5-model ensemble.
• The single-model F1 score reaches 67.8%, outperforming prior state-of-the-art models by improving precision, particularly on long spans.
• Ablation studies show that removing the mention detection loss or biaffine attention reduces F1 by 0.3–0.4 points, confirming both components' contributions.
• The model detects 1,048 mentions not identified by Lee et al. (2017), including 662 that are absent from the training data, demonstrating improved generalization.
• Mention detection performance improves significantly for spans of 5 or more words, with higher accuracy across all widths due to the explicit detection loss.
• The biaffine attention mechanism enhances antecedent scoring, leading to better coreference linking and higher precision in clustering.