[Paper Review] Neural Collective Entity Linking

Yixin Cao, Lei Hou+2 more|arXiv (Cornell University)|Nov 21, 2018

Topic Modeling34 references61 citations

TL;DR

NCEL develops a neural collective entity linking model that uses Graph Convolutional Networks on subgraphs with an attention mechanism, trained on Wikipedia hyperlinks, and achieves strong generalization across five benchmarks.

ABSTRACT

Entity Linking aims to link entity mentions in texts to knowledge bases, and neural models have achieved recent success in this task. However, most existing methods rely on local contexts to resolve entities independently, which may usually fail due to the data sparsity of local information. To address this issue, we propose a novel neural model for collective entity linking, named as NCEL. NCEL applies Graph Convolutional Network to integrate both local contextual features and global coherence information for entity linking. To improve the computation efficiency, we approximately perform graph convolution on a subgraph of adjacent entity mentions instead of those in the entire text. We further introduce an attention scheme to improve the robustness of NCEL to data noise and train the model on Wikipedia hyperlinks to avoid overfitting and domain bias. In experiments, we evaluate NCEL on five publicly available datasets to verify the linking performance as well as generalization ability. We also conduct an extensive analysis of time complexity, the impact of key modules, and qualitative results, which demonstrate the effectiveness and efficiency of our proposed method.

Motivation & Objective

• Motivate the need for collective entity linking to overcome local-context sparsity in disambiguation.
• Propose a neural model that combines local contextual features with global coherence through a subgraph GCN.
• Enable end-to-end training by integrating candidate features, subgraph structure, and attention.
• Improve efficiency by performing graph convolutions on adjacent-mentions subgraphs rather than the full graph.

Proposed method

• Generate candidate entities for each mention using a prior-based dictionary sourced from Wikipedia, web corpora, and YAGO.
• Extract both local features (string similarity, compatibility via joint embeddings with attention) and global features (neighbor mention compatibility and subgraph structure).
• Represent mentions and candidates in a joint embedding space with sense-aware representations for words/mentions and entities.
• Apply a multi-layer encoder, followed by a sub-graph convolutional network that operates on a truncated subgraph of adjacent mentions, and a decoder to predict candidate probabilities.
• Train end-to-end by minimizing cross-entropy between predicted and ground-truth candidates across documents.
• Use an attention mechanism to weight contextual words contributing to candidate compatibility, improving robustness to noise.

Experimental results

Research questions

• RQ1 Does NCEL achieve better linking accuracy than state-of-the-art local and global models across diverse datasets?
• RQ2 Can a subgraph-based GCN effectively capture document-level coherence without full graph computation?
• RQ3 How do the attention mechanism and joint embeddings impact robustness to noisy data and domain bias?
• RQ4 What is the relative contribution of local versus global features in NCEL's performance?
• RQ5 How well does training on Wikipedia hyperlinks generalize to standard EL benchmarks?

Key findings

• NCEL consistently outperforms a range of baselines on five public EL benchmarks with strong generalization.
• Using subgraphs of adjacent mentions significantly improves efficiency while maintaining competitive performance.
• The attention mechanism enhances robustness to noise and improves feature quality for disambiguation.
• Joint word/entity embeddings enable effective integration of local and global cues within an end-to-end framework.
• Training on Wikipedia hyperlinks yields strong generalization across datasets and domains.
• Qualitative analysis shows NCEL benefits from neighbor mention signals to disambiguate challenging cases.