[Paper Review] Studying the Wikipedia Hyperlink Graph for Relatedness and Disambiguation

Eneko Agirre, Ander Barrena+1 more|arXiv (Cornell University)|Mar 5, 2015

Wikis in Education and Collaboration40 references19 citations

TL;DR

This paper proposes using full Wikipedia hyperlink graphs—via personalized PageRank random walks—instead of just direct links for word relatedness and named-entity disambiguation. It demonstrates that traversing the entire graph significantly outperforms direct-link methods, with non-reciprocal links harming performance and categories/infoboxes contributing little. The approach sets a new state-of-the-art for Wikipedia-link-based systems, matching or exceeding multi-source and supervised methods.

ABSTRACT

Hyperlinks and other relations in Wikipedia are a extraordinary resource which is still not fully understood. In this paper we study the different types of links in Wikipedia, and contrast the use of the full graph with respect to just direct links. We apply a well-known random walk algorithm on two tasks, word relatedness and named-entity disambiguation. We show that using the full graph is more effective than just direct links by a large margin, that non-reciprocal links harm performance, and that there is no benefit from categories and infoboxes, with coherent results on both tasks. We set new state-of-the-art figures for systems based on Wikipedia links, comparable to systems exploiting several information sources and/or supervised machine learning. Our approach is open source, with instruction to reproduce results, and amenable to be integrated with complementary text-based methods.

Motivation & Objective

• To evaluate the effectiveness of full hyperlink graphs versus direct links in Wikipedia for semantic tasks.
• To analyze the impact of different link sources—such as non-reciprocal links, categories, and infoboxes—on performance.
• To establish a clear, reproducible benchmark for Wikipedia-based random walk methods in relatedness and disambiguation.
• To achieve state-of-the-art results using only Wikipedia hyperlinks, without external knowledge sources or supervised learning.
• To provide open-source tools and instructions for full replicability of results.

Proposed method

• The authors apply personalized PageRank on Wikipedia hyperlink graphs to compute semantic relatedness and disambiguation scores.
• They compare the full graph (all reachable paths) against only direct hyperlinks, using a consistent random walk framework.
• They evaluate multiple graph variants: including reciprocal vs. non-reciprocal links, category links, and infobox links.
• The method uses a single, well-established algorithm (Personalized PageRank) across all experiments to ensure fair comparison.
• Results are validated on diverse, standard datasets for word relatedness and named-entity disambiguation.
• The system is open-sourced with full documentation for reproducibility.

Experimental results

Research questions

• RQ1 Does using the full hyperlink graph in Wikipedia improve performance over using only direct hyperlinks in semantic tasks?
• RQ2 How do different types of links—reciprocal, non-reciprocal, category-based, and infobox links—affect the performance of random walk-based methods?
• RQ3 What is the relative contribution of Wikipedia’s category structure and infoboxes to relatedness and disambiguation tasks?
• RQ4 Can a simple, unsupervised random walk on Wikipedia hyperlinks achieve state-of-the-art performance without external knowledge or supervised learning?
• RQ5 Is the performance gain from using the full graph consistent across multiple datasets and tasks?

Key findings

• Using the full hyperlink graph via personalized PageRank significantly outperforms using only direct links, with consistent improvements across all datasets.
• Non-reciprocal links degrade performance, indicating that asymmetric link structures introduce noise.
• Category links and infobox links contribute minimally to performance, showing little to no benefit in the tested tasks.
• The method achieves new state-of-the-art results on both word relatedness and named-entity disambiguation benchmarks using only Wikipedia hyperlinks.
• The system's performance is comparable to top systems that use multiple information sources or supervised learning techniques.
• Combining Wikipedia-based random walks with WordNet further improves results, especially in relatedness tasks.