[Paper Review] Learning Entity Linking Features for Emerging Entities

Chenwei Ran, Wei Shen+4 more|arXiv (Cornell University)|Aug 8, 2022

Topic Modeling4 citations

TL;DR

This paper proposes STAMO, a self-training approach that learns high-quality entity linking (EL) features for emerging entities (EEs) lacking Wikipedia coverage. By treating self-training as a multi-slot optimization process with intra- and inter-slot refinements, STAMO mitigates error propagation and significantly improves EL performance on EEs using only a few labeled documents and unlabeled web data.

ABSTRACT

Entity linking (EL) is the process of linking entity mentions appearing in text with their corresponding entities in a knowledge base. EL features of entities (e.g., prior probability, relatedness score, and entity embedding) are usually estimated based on Wikipedia. However, for newly emerging entities (EEs) which have just been discovered in news, they may still not be included in Wikipedia yet. As a consequence, it is unable to obtain required EL features for those EEs from Wikipedia and EL models will always fail to link ambiguous mentions with those EEs correctly as the absence of their EL features. To deal with this problem, in this paper we focus on a new task of learning EL features for emerging entities in a general way. We propose a novel approach called STAMO to learn high-quality EL features for EEs automatically, which needs just a small number of labeled documents for each EE collected from the Web, as it could further leverage the knowledge hidden in the unlabeled data. STAMO is mainly based on self-training, which makes it flexibly integrated with any EL feature or EL model, but also makes it easily suffer from the error reinforcement problem caused by the mislabeled data. Instead of some common self-training strategies that try to throw the mislabeled data away explicitly, we regard self-training as a multiple optimization process with respect to the EL features of EEs, and propose both intra-slot and inter-slot optimizations to alleviate the error reinforcement problem implicitly. We construct two EL datasets involving selected EEs to evaluate the quality of obtained EL features for EEs, and the experimental results show that our approach significantly outperforms other baseline methods of learning EL features.

Motivation & Objective

• Address the challenge of entity linking (EL) failure for emerging entities (EEs) not yet in Wikipedia, which lack essential EL features like prior probability and embeddings.
• Overcome the limitations of manual annotation and existing retrieval-based methods that fail to generate strong EL features for EEs.
• Develop a general-purpose method to automatically learn high-quality EL features for EEs using only a small number of labeled documents and abundant unlabeled web data.
• Mitigate the error reinforcement problem in self-training by implicitly refining EL features through structured optimization, rather than discarding mislabeled data.

Proposed method

• Propose STAMO, a self-training framework that learns EL features for EEs from a small set of labeled documents and large-scale unlabeled web data.
• Treat self-training as a multiple optimization process over EL features of EEs, introducing intra-slot optimization to align EL features with the objective function of the base EL model.
• Introduce inter-slot optimization to connect current EL feature learning with historical learning slots, enabling knowledge transfer across iterations.
• Use a hypothetical optimization process to model the evolution of EL features over time, improving robustness and reducing error accumulation.
• Integrate seamlessly with any numerical EL feature (e.g., prior probability, relatedness score, entity embedding) or EL model architecture.
• Leverage the structure of the learning process to implicitly suppress noisy predictions without explicit instance filtering.

Experimental results

Research questions

• RQ1 Can self-training be effectively adapted to learn high-quality EL features for emerging entities when Wikipedia-based features are unavailable?
• RQ2 How can the error reinforcement problem in self-training be mitigated without discarding mislabeled data?
• RQ3 To what extent can intra-slot and inter-slot optimization improve the quality of learned EL features for EEs?
• RQ4 Can a small number of labeled documents combined with unlabeled data achieve performance comparable to or better than supervised baselines?
• RQ5 How does the proposed optimization strategy compare to standard self-training or confidence-based filtering in terms of robustness and accuracy for EEs?

Key findings

• STAMO significantly outperforms baseline methods in learning EL features for emerging entities, demonstrating superior performance on two constructed EL datasets.
• The proposed intra-slot optimization effectively aligns learned EL features with the objective function of the base EL model, improving feature quality.
• Inter-slot optimization enhances learning by leveraging historical feature updates, reducing error propagation across iterations.
• The method achieves high-quality EL feature learning with minimal supervision—only a few labeled documents per emerging entity are required.
• STAMO’s implicit error mitigation strategy via optimization outperforms explicit mislabeled data rejection, proving more robust in practice.
• The approach is generalizable and can be flexibly integrated with any numerical EL feature or EL model architecture.