[Paper Review] NeMig -- A Bilingual News Collection and Knowledge Graph about Migration

Andreea Iana, Mehwish Alam+6 more|arXiv (Cornell University)|Sep 1, 2023

Topic ModelingComputer Science3 citations

TL;DR

NeMig introduces a bilingual (German/English) news collection and knowledge graph focused on migration, annotated with sentiment polarization, media political leanings, subtopics, and Wikidata-disambiguated entities. It includes anonymized user data with socio-demographic and political attributes, enabling analysis of recommender biases, filter bubbles, and cross-lingual news curation effects beyond accuracy.

ABSTRACT

News recommendation plays a critical role in shaping the public's worldviews through the way in which it filters and disseminates information about different topics. Given the crucial impact that media plays in opinion formation, especially for sensitive topics, understanding the effects of personalized recommendation beyond accuracy has become essential in today's digital society. In this work, we present NeMig, a bilingual news collection on the topic of migration, and corresponding rich user data. In comparison to existing news recommendation datasets, which comprise a large variety of monolingual news, NeMig covers articles on a single controversial topic, published in both Germany and the US. We annotate the sentiment polarization of the articles and the political leanings of the media outlets, in addition to extracting subtopics and named entities disambiguated through Wikidata. These features can be used to analyze the effects of algorithmic news curation beyond accuracy-based performance, such as recommender biases and the creation of filter bubbles. We construct domain-specific knowledge graphs from the news text and metadata, thus encoding knowledge-level connections between articles. Importantly, while existing datasets include only click behavior, we collect user socio-demographic and political information in addition to explicit click feedback. We demonstrate the utility of NeMig through experiments on the tasks of news recommenders benchmarking, analysis of biases in recommenders, and news trends analysis. NeMig aims to provide a useful resource for the news recommendation community and to foster interdisciplinary research into the multidimensional effects of algorithmic news curation.

Motivation & Objective

• Address the lack of multilingual, topic-specific news datasets with political and sentiment annotations for controversial topics like migration.
• Enable research into algorithmic recommender biases and filter bubble formation beyond accuracy-based evaluation.
• Provide a resource for interdisciplinary study of media curation effects on political polarization and public opinion.
• Support the development of knowledge-aware recommenders through structured, Wikidata-enhanced knowledge graphs.
• Foster synthetic user data generation by including rich user profiles alongside explicit click feedback.

Proposed method

• Curated 7,000 German and 10,000 U.S. news articles from diverse media outlets spanning the political spectrum.
• Annotated articles with sentiment polarization and media outlet political leanings using expert labeling and existing classification schemes.
• Extracted subtopics and named entities, disambiguating them via Wikidata to ensure semantic consistency.
• Constructed domain-specific knowledge graphs (NeMigKG) by integrating news text, metadata, and up to two-hop neighbors from Wikidata.
• Pre-trained entity embeddings using TransD on multiple NeMigKG variants to evaluate impact on recommendation performance.
• Collected anonymized user data including demographics, political attitudes, personality traits, and explicit click feedback for 3,000 users per language.

(a) German news corpus.

Experimental results

Research questions

• RQ1 How does incorporating political leanings and sentiment polarization in news articles affect recommender system performance and diversity?
• RQ2 To what extent do knowledge graph extensions with Wikidata neighbors improve recommendation accuracy and aspect-based diversity?
• RQ3 How do media outlets with different political orientations cover the same migration-related entities over time?
• RQ4 Can user socio-demographic and political data help detect and analyze filter bubble effects in news recommendation systems?
• RQ5 What are the temporal trends in entity coverage across political media outlets in Germany and the U.S.?

Key findings

• Adding political leaning, sentiment, and subtopic information to NeMigKG improved aspect-based diversity in recommendations without significantly affecting accuracy.
• Extending NeMigKG with one-hop Wikidata neighbors improved recommendation performance, but two-hop neighbors reduced performance, indicating knowledge overloading in small datasets.
• The top-20 most frequently mentioned entities in German media were predominantly covered by center-right and center media, with notable exceptions like Angela Merkel and Die Linke appearing more frequently in right-wing outlets.
• In U.S. media, left-leaning outlets reported less frequently on the most common entities compared to center or right-leaning outlets.
• Temporal analysis revealed that entity coverage in German media correlated with major political events such as federal and European elections or international conflicts like the Turkish offensive in northern Syria.
• The inclusion of user demographic and political data enabled detection of user-specific bias patterns, supporting the study of personalized recommendation effects on political polarization.

(b) English news corpus.