[Paper Review] WikiDataSets: Standardized sub-graphs from Wikidata

Armand Boschin, Bonald, Thomas|arXiv (Cornell University)|Jun 11, 2019

Advanced Graph Neural Networks11 references4 citations

TL;DR

This paper introduces WikiDataSets, a collection of standardized, topic-specific subgraphs extracted from Wikidata, enabling reproducible research in knowledge graph processing and relational learning. Using a reusable Python package, the authors generate five datasets—animals, companies, countries, films, and humans—by recursively collecting sub-classes and filtering relevant facts, with evaluation showing strong performance in link prediction using TransH and ANALOGY models on the humans dataset.

ABSTRACT

Developing new ideas and algorithms in the fields of graph processing and relational learning requires public datasets. While Wikidata is the largest open source knowledge graph, involving more than fifty million entities, it is larger than needed in many cases and even too large to be processed easily. Still, it is a goldmine of relevant facts and relations. Using this knowledge graph is time consuming and prone to task specific tuning which can affect reproducibility of results. Providing a unified framework to extract topic-specific subgraphs solves this problem and allows researchers to evaluate algorithms on common datasets. This paper presents various topic-specific subgraphs of Wikidata along with the generic Python code used to extract them. These datasets can help develop new methods of knowledge graph processing and relational learning.

Motivation & Objective

• To address the challenge of inconsistent, large-scale, and hard-to-process Wikidata for relational learning and graph algorithm evaluation.
• To reduce reproducibility issues caused by ad-hoc dataset curation in existing benchmarking workflows.
• To provide a unified, open-source framework for extracting and using topic-specific subgraphs from Wikidata.
• To enable researchers to evaluate new algorithms on consistent, well-documented, and publicly available datasets.

Proposed method

• The WikiDataSets Python package extracts subgraphs by identifying all Wikidata entities that are instances of a topic or its sub-classes (e.g., Q5 for humans).
• It processes the full Wikidata dump (latest-all.json.bz2) to retain only facts involving selected nodes, preserving labeled relations and attributes.
• The pipeline includes three main steps: retrieving sub-classes of a topic, filtering facts from the dump based on selected entities, and building structured output files.
• Output includes five files per dataset: edges.txt (triplets), attributes.txt (external facts), entities.txt (labels and Wikidata IDs), nodes.txt (subset of entities), and relations.txt (relation metadata).
• The framework supports label collection in English and ensures all graphs remain fully labeled knowledge graphs.
• The code is open-source and available on PyPI and the project’s website for reuse and extension.

Experimental results

Research questions

• RQ1 Can standardized, topic-specific subgraphs from Wikidata be automatically generated to improve reproducibility in knowledge graph research?
• RQ2 How do existing knowledge graph embedding models perform on curated Wikidata subgraphs compared to traditional benchmarks?
• RQ3 To what extent do the structural and relational properties of Wikidata subgraphs support meaningful community detection and semantic analysis?
• RQ4 Can a generic, reusable framework streamline the extraction of domain-specific knowledge graphs from large-scale open knowledge bases like Wikidata?

Key findings

• The humans dataset, filtered to include only entities with more than five facts, contains 238,376 nodes and 722,993 edges, forming a high-quality knowledge graph for evaluation.
• On the filtered humans dataset, the ANALOGY model achieved a filtered Hit@10 of 67.9 and a filtered Mean Rank of 10,147, outperforming TransH.
• TransH achieved a filtered Hit@10 of 61.8 and a filtered Mean Rank of 15,027, indicating strong but slightly lower performance than ANALOGY.
• Community detection using the Louvain algorithm on the humans dataset revealed semantically coherent clusters, such as American artists and Vietnamese political leaders, visualized via 3D force-directed layout.
• The top edge types in each dataset reflect domain-specific relations—e.g., 'child', 'spouse', and 'father' dominate in the humans dataset, while 'parent organization' and 'subsidiary' are prevalent in companies.
• The WikiDataSets package successfully automates the creation of consistent, labeled, and reusable subgraphs, with all datasets and code publicly available for research use.