[Paper Review] SWAT: A System for Detecting Salient Wikipedia Entities in Texts

Marco Ponza, Paolo Ferragina+1 more|arXiv (Cornell University)|Apr 10, 2018

Topic Modeling55 references6 citations

TL;DR

SWAT is a novel system for detecting salient Wikipedia entities in text by leveraging a pipeline of syntactic, semantic, and latent features—such as dependency trees, entity embeddings, and graph centrality measures—trained on millions of examples from the New York Times corpus. It achieves state-of-the-art performance, improving F1 scores by up to 6.3 percentage points over prior systems like Cmu-Google and Sel.

ABSTRACT

We study the problem of entity salience by proposing the design and implementation of SWAT, a system that identifies the salient Wikipedia entities occurring in an input document. SWAT consists of several modules that are able to detect and classify on-the-fly Wikipedia entities as salient or not, based on a large number of syntactic, semantic and latent features properly extracted via a supervised process which has been trained over millions of examples drawn from the New York Times corpus. The validation process is performed through a large experimental assessment, eventually showing that SWAT improves known solutions over all publicly available datasets. We release SWAT via an API that we describe and comment in the paper in order to ease its use in other software.

Motivation & Objective

• To address the challenge of identifying the most salient Wikipedia entities in a given text document, which is critical for downstream NLP and IR applications.
• To improve upon existing entity salience detection systems by integrating a richer set of syntactic, semantic, and latent features derived from multiple NLP components.
• To provide a robust, publicly accessible system that enhances entity salience detection for use in information retrieval, summarization, and knowledge extraction tasks.
• To conduct a comprehensive evaluation of entity salience systems across two major datasets—New York Times and Wikinews—offering a fair comparison of state-of-the-art methods.
• To release SWAT as a public API to facilitate integration into other software and encourage further research in entity salience and linking.

Proposed method

• The system processes input text through a three-module pipeline: Document Enrichment, Feature Generation, and Entity Salience Classification.
• Document Enrichment uses CoreNLP to extract syntactic structures like dependency trees and sentence-level keyphrases via TextRank.
• It employs Wat, a high-accuracy entity linker, to identify and link mentions to Wikipedia entities, constructing an entity graph with semantic relationships.
• Word2Vec embeddings are used to enrich the entity graph with distributional semantic signals, capturing latent semantic similarity between entities.
• Feature Generation maps each detected entity into a comprehensive vector combining syntactic (e.g., sentence rank, dependency path), semantic (e.g., graph centrality), and latent (e.g., word and entity embeddings) features.
• The Entity Salience Classification module uses a supervised machine learning model trained on millions of annotated examples to classify each entity as salient or non-salient based on its feature vector.

Experimental results

Research questions

• RQ1 How do different combinations of syntactic, semantic, and latent features impact the performance of entity salience detection?
• RQ2 To what extent can entity embeddings and graph-based centrality measures improve the identification of salient entities compared to traditional frequency and position-based signals?
• RQ3 How does SWAT compare to existing state-of-the-art systems like Cmu-Google and Sel in terms of F1 score across diverse datasets?
• RQ4 What is the impact of feature engineering and model design on the efficiency and accuracy of salience classification, especially in large-scale text processing?
• RQ5 Can a publicly available API for end-to-end entity linking and salience detection significantly lower the barrier for adoption in downstream NLP applications?

Key findings

• SWAT achieves a 6.3 percentage point absolute improvement in F1 score over the Sel system and a 3.4 percentage point improvement over the Cmu-Google system on the New York Times and Wikinews datasets.
• On the largest dataset, New York Times, SWAT improves micro-F1 by up to 14% compared to Cmu-Google, demonstrating strong performance on long documents.
• The system's performance is significantly enhanced by the integration of word and entity embeddings, which provide crucial latent signals beyond surface-level frequency and position.
• Ablation studies confirm that all components—syntactic, semantic, and latent features—are essential for the system’s high performance, with each contributing uniquely to the final classification.
• The release of SWAT as a public API enables seamless integration into other NLP pipelines, promoting reuse and extension of the system in academic and industrial applications.
• The study identifies key limitations in existing datasets, such as the rule-based ground-truth labeling in the New York Times dataset, and suggests improvements via crowdsourcing and enhanced annotation.