[Paper Review] Neural Cross-Lingual Entity Linking

Avirup Sil, Gourab Kundu+2 more|arXiv (Cornell University)|Dec 5, 2017

Topic Modeling51 citations

TL;DR

An English-trained neural EL model learns fine-grained context similarities to disambiguate mentions, and uses multilingual embeddings to enable zero-shot cross-lingual EL for Spanish and Chinese TAC 2015 datasets, achieving state-of-the-art results.

ABSTRACT

A major challenge in Entity Linking (EL) is making effective use of contextual information to disambiguate mentions to Wikipedia that might refer to different entities in different contexts. The problem exacerbates with cross-lingual EL which involves linking mentions written in non-English documents to entries in the English Wikipedia: to compare textual clues across languages we need to compute similarity between textual fragments across languages. In this paper, we propose a neural EL model that trains fine-grained similarities and dissimilarities between the query and candidate document from multiple perspectives, combined with convolution and tensor networks. Further, we show that this English-trained system can be applied, in zero-shot learning, to other languages by making surprisingly effective use of multi-lingual embeddings. The proposed system has strong empirical evidence yielding state-of-the-art results in English as well as cross-lingual: Spanish and Chinese TAC 2015 datasets.

Motivation & Objective

• Develop a neural EL model that learns fine-grained similarities between query context and candidate Wikipedia pages.
• Enable zero-shot cross-lingual EL using multilingual embeddings without retraining.
• Integrate multiple representations of context (CNNs, LSTMs, NTN) and coherence features to improve disambiguation.
• Utilize fast anchor-title-based candidate retrieval and cross-language mapping via inter-language links.
• Evaluate on English (CoNLL/TAC) and cross-lingual TAC 2015 Spanish/Chinese datasets to establish state-of-the-art performance.

Proposed method

• Fast match candidate generation using an anchor-title index built from English and target-language Wikipedias, mapped via inter-language links.
• Embed words with multilingual embeddings (MultiCCA, CCA, LS) and represent Wikipedia pages as IDF-weighted word embeddings.
• Model context with CNNs over sentences in the coreference chain of the mention and first paragraphs of candidate pages.
• Apply fine-grained context modeling with left/right contexts using LSTMs and neural tensor networks to capture nuanced Dis/Similarities.
• Construct a feature abstraction layer with multiple similarity measures (Sentence context–Wiki link, Sentence context–Wiki first paragraph, Fine-grained context–Wiki link) plus LIEL-style features, processed by a feed-forward network and trained as a binary classifier.
• Train with positive/negative example pairs and optimize a cross-entropy loss; decoding uses fast-match candidates and the trained scorer to select the best link or NIL.

Experimental results

Research questions

• RQ1 Can an EL model trained only on English perform cross-lingual linking in other languages without retraining (zero-shot)?
• RQ2 Which multilingual embedding strategy (MultiCCA, CCA, LS) best supports cross-lingual EL for Spanish and Chinese?
• RQ3 Do lexical decomposition/composition (LDC) and multi-perspective context matching (MPCM) layers improve EL performance?
• RQ4 How does the proposed zero-shot cross-lingual EL approach compare to state-of-the-art systems on English and cross-lingual TAC datasets?

Key findings

• The model achieves state-of-the-art results on English EL benchmarks (CoNLL 2003 and TAC 2010).
• In cross-lingual EL, the approach attains state-of-the-art performance on TAC 2015 Spanish and Chinese datasets.
• Among multilingual embeddings, MultiCCA-based representations outperform LS and CCA for cross-lingual EL.
• Incorporating LDC and MPCM layers provides measurable performance gains over baselines and simpler architectures.
• The full model with context LSTMs, LDC, and MPCM yields the best reported results compared to prior work in both monolingual and cross-lingual settings.