[Paper Review] A Large-Scale Multilingual Disambiguation of Glosses

José Camacho-Collados, Claudio Delli Bovi+2 more|arXiv (Cornell University)|Aug 24, 2016

Natural Language Processing Techniques26 references3 citations

TL;DR

This paper presents a large-scale multilingual corpus of disambiguated textual definitions (glosses) from multiple sources and 263 languages, leveraging BabelNet’s semantic network and cross-lingual complementarity to achieve over 90% disambiguation precision. The corpus, freely available at http://lcl.uniroma1.it/disambiguated-glosses, significantly improves performance in Open Information Extraction and Sense Clustering tasks when integrated into state-of-the-art systems.

ABSTRACT

Linking concepts and named entities to knowledge bases has become a crucial Natural Language Understanding task. In this respect, recent works have shown the key advantage of exploiting textual definitions in various Natural Language Processing applications. However, to date there are no reliable large-scale corpora of sense-annotated textual definitions available to the research community. In this paper we present a large-scale high-quality corpus of disambiguated glosses in multiple languages, comprising sense annotations of both concepts and named entities from a unified sense inventory. Our approach for the construction and disambiguation of the corpus builds upon the structure of a large multilingual semantic network and a state-of-the-art disambiguation system; first, we gather complementary information of equivalent definitions across different languages to provide context for disambiguation, and then we combine it with a semantic similarity-based refinement. As a result we obtain a multilingual corpus of textual definitions featuring over 38 million definitions in 263 languages, and we make it freely available at http://lcl.uniroma1.it/disambiguated-glosses. Experiments on Open Information Extraction and Sense Clustering show how two state-of-the-art approaches improve their performance by integrating our disambiguated corpus into their pipeline.

Motivation & Objective

• To address the lack of large-scale, high-quality, sense-annotated corpora of textual definitions for multilingual NLP applications.
• To improve the reliability and coverage of definitional knowledge by exploiting cross-lingual and cross-resource complementarities in definitions.
• To construct a disambiguated gloss corpus that supports both high-coverage and high-precision NLP pipelines.
• To evaluate the utility of the corpus in intrinsic and extrinsic NLP tasks, demonstrating its impact on downstream systems.

Proposed method

• Leveraging BabelNet, a multilingual semantic network, to unify sense inventories and provide a large-scale target for disambiguation.
• Gathering equivalent definitions across languages and resources to enrich context for disambiguation.
• Applying a semantic similarity-based refinement module using NASARI vectors to improve disambiguation precision.
• Using Babelfy and heuristic methods (MCS) to generate initial disambiguation scores, then filtering for high-precision instances.
• Storing results in standardized XML format, with per-definition annotations including BabelNet IDs, source, surface form match, and confidence scores.
• Releasing both a complete corpus and a high-precision subset, with the latter optimized for low-error, high-accuracy applications.

Experimental results

Research questions

• RQ1 Can cross-lingual and cross-resource complementarities in textual definitions significantly improve disambiguation performance?
• RQ2 What level of disambiguation precision can be achieved on a multilingual, multi-source corpus of definitions?
• RQ3 How effective is the disambiguated gloss corpus in improving downstream NLP tasks such as Open Information Extraction and Sense Clustering?
• RQ4 Can a high-precision subset of the corpus serve as a reliable semantic network or knowledge base enhancement?

Key findings

• The disambiguation system achieved over 90% precision on a random sample of definitions across three languages, outperforming prior approaches.
• The high-precision version of the corpus, enriched with NASARI-based semantic similarity, improved F1 scores by 7.3 points on the SemEval Wikipedia sense clustering dataset.
• Integrating the disambiguated glosses into an Open Information Extraction system led to measurable performance gains over the original pipeline.
• The NASARI+glosses model achieved 88.1% accuracy on the SemEval dataset, outperforming both the baseline and the Dandala-multilingual system.
• The complete corpus contains over 38 million disambiguated definitions across 263 languages, making it the largest known multilingual gloss corpus to date.
• The high-precision corpus, while smaller in size, offers significantly reduced error rates, making it suitable for high-accuracy applications such as semantic network enrichment.