[Paper Review] Two Tales of the World: Comparison of Widely Used World News Datasets GDELT and EventRegistry

Haewoon Kwak, Jisun An|arXiv (Cornell University)|Mar 7, 2016

Data Management and Algorithms12 citations

TL;DR

This paper compares GDELT and EventRegistry, two major world news datasets, across scale, news sources, and news geography. Despite significant differences in volume and source diversity—GDELT indexes 2.26 to 6.43 times more articles than EventRegistry and covers 64.1 languages versus 14—news geography, measured by country mentions, shows a high correlation (Spearman’s ρ = 0.867), indicating consistent global news coverage patterns despite underlying data disparities.

ABSTRACT

In this work, we compare GDELT and Event Registry, which monitor news articles worldwide and provide big data to researchers regarding scale, news sources, and news geography. We found significant differences in scale and news sources, but surprisingly, we observed high similarity in news geography between the two datasets.

Motivation & Objective

• To evaluate and compare the scale, news source diversity, and geographic coverage of two widely used world news datasets: GDELT and EventRegistry.
• To assess the implications of dataset differences for researchers using these resources in computational journalism and social science research.
• To investigate whether divergent data collection methods result in divergent global news representations, particularly in terms of country-level media attention.
• To provide empirical guidance on dataset selection and usage, emphasizing the importance of understanding data provenance and limitations.

Proposed method

• The study samples the first day of each month from March to December 2015 to avoid temporal fluctuations and ensure comparability.
• GDELT data is collected via compressed daily dump files (English and Translingual), while EventRegistry data is retrieved using its public API with pagination and date limits.
• News geography is quantified by counting the number of articles mentioning each country, using the V2ENHANCEDLOCATIONS field in GDELT and location metadata in EventRegistry.
• Correlation analysis (Spearman’s rank correlation) is used to compare the news geography of both datasets, measuring similarity in country-level media attention.
• Source-level analysis compares the number of unique websites, language distribution, and article volume per source between the two platforms.
• Statistical tests, including Pearson correlation and Spearman’s rho, are applied to assess relationships between article volume trends and source characteristics.

Experimental results

Research questions

• RQ1 How do the scale and source diversity of GDELT and EventRegistry compare across the same time period?
• RQ2 To what extent do the two datasets differ in language coverage and the distribution of articles across languages?
• RQ3 How similar are the news geographies—measured by country-level article mentions—between GDELT and EventRegistry?
• RQ4 What is the relationship between a news source’s web traffic and its output volume in each dataset?
• RQ5 How do the temporal patterns of article volume differ between the two platforms, and what factors drive these differences?

Key findings

• GDELT indexes 2.26 to 6.43 times more articles per day than EventRegistry, with a Pearson correlation of r = 0.57 (p = 0.08) between their daily article volumes.
• GDELT covers 64.1 languages on average per day, while EventRegistry indexes only 14 languages, though the top 10 languages are largely the same across both datasets.
• Despite differences in source selection, the news geography of both datasets shows a high correlation (Spearman’s ρ = 0.867, p = 1.896e-74), indicating consistent global media attention patterns.
• The top 10% of news sources in both datasets produce about 80% of all articles, but there is no significant correlation between a source’s web traffic and its article output volume (ρ = 0.17, p = 0.10).
• While GDELT claims to collect from broadcast, print, and offline sources, 99.9% of its articles are from the web, and only 0.1% come from non-web sources, primarily BBC Monitoring.
• The two datasets share 13,867 out of 63,268 GDELT sources and 20,754 EventRegistry sources, indicating limited overlap despite both focusing on web-based news.