[Paper Review] Evaluating the word-expert approach for Named-Entity Disambiguation

Anne Lynn S. Chang, Valentin I. Spitkovsky+2 more|arXiv (Cornell University)|Mar 15, 2016

Natural Language Processing Techniques30 references3 citations

TL;DR

This paper evaluates the word-expert approach—training a separate classifier for each named-entity mention string—for Named-Entity Disambiguation (NED) using automatically extracted Wikipedia anchor texts. It demonstrates that this method, adapted from word-sense disambiguation, achieves competitive performance on TAC 2009 and 2010 datasets, showing that supervised classification with entity-specific models is effective and feasible for NED despite higher ambiguity and synonymy in real-world data.

ABSTRACT

Named Entity Disambiguation (NED) is the task of linking a named-entity mention to an instance in a knowledge-base, typically Wikipedia. This task is closely related to word-sense disambiguation (WSD), where the supervised word-expert approach has prevailed. In this work we present the results of the word-expert approach to NED, where one classifier is built for each target entity mention string. The resources necessary to build the system, a dictionary and a set of training instances, have been automatically derived from Wikipedia. We provide empirical evidence of the value of this approach, as well as a study of the differences between WSD and NED, including ambiguity and synonymy statistics.

Motivation & Objective

• To investigate whether the word-expert approach—commonly used in word-sense disambiguation (WSD)—can be effectively applied to Named-Entity Disambiguation (NED).
• To evaluate the performance of entity-specific classifiers trained on automatically extracted Wikipedia anchor-texts for NED.
• To analyze the differences between WSD and NED in terms of ambiguity, synonymy, and data distribution.
• To assess whether supervised classification techniques from WSD can serve as strong, replicable baselines for NED systems.

Proposed method

• The system uses a two-stage architecture: a context-independent module based on entity frequency (most frequent sense heuristic) and a context-sensitive classifier per named-entity string.
• For each entity mention string, a logistic regression classifier is trained using anchor-texts from Wikipedia as training instances.
• Training data is automatically derived from Wikipedia by extracting hyperlinked anchor-texts pointing to specific entity pages.
• The system does not require dynamic candidate generation; instead, it relies on a pre-built dictionary of entity mappings from strings to Wikipedia pages.
• The approach is designed to be fully replicable using only offline resources, enabling clear ablation and modular extension.
• The method is evaluated on the TAC-KBP 2009 and 2010 datasets without test-set parameter tuning or thresholds.

Experimental results

Research questions

• RQ1 Can the word-expert approach from WSD be successfully adapted to NED using automatically extracted Wikipedia data?
• RQ2 How does the performance of entity-specific classifiers compare to state-of-the-art NED systems on standard benchmarks?
• RQ3 What are the key differences in ambiguity and synonymy between WSD and NED, and how do they affect disambiguation performance?
• RQ4 Why might NED be easier in practice than WSD despite higher theoretical ambiguity and synonymy?
• RQ5 Can a simple, modular system based on supervised classifiers and frequency heuristics serve as a strong baseline for NED?

Key findings

• The word-expert approach achieves competitive performance on the TAC 2009 and 2010 NED benchmarks, demonstrating its viability for NED.
• Despite higher polysemy and synonymy in NED, the method performs well due to strong data skew toward popular entities and high inter-annotator agreement.
• The system's performance is robust without test-set tuning, indicating generalization across diverse mention contexts.
• The use of anchor-texts as training data enables automatic, scalable, and reproducible system construction.
• The results suggest that NED is practically easier than WSD due to richer training data and clearer semantic distinctions in real-world mentions.
• The approach provides a clear, modular baseline that can be extended with additional components like global coherence or candidate generation.