[Paper Review] Polarization and Fake News: Early Warning of Potential Misinformation Targets

Michela Del Vicario, Walter Quattrociocchi+2 more|arXiv (Cornell University)|Feb 5, 2018

Misinformation and Its ImpactsSocial Sciences29 references60 citations

TL;DR

The authors present a general framework to identify polarizing content on social media to pre-emptively flag topics likely to be misused in fake news, validated on Italian Facebook data with 77% topic-level accuracy and 91% accuracy when used as a feature for fake-news detection.

ABSTRACT

Users polarization and confirmation bias play a key role in misinformation spreading on online social media. Our aim is to use this information to determine in advance potential targets for hoaxes and fake news. In this paper, we introduce a general framework for promptly identifying polarizing content on social media and, thus, "predicting" future fake news topics. We validate the performances of the proposed methodology on a massive Italian Facebook dataset, showing that we are able to identify topics that are susceptible to misinformation with 77% accuracy. Moreover, such information may be embedded as a new feature in an additional classifier able to recognize fake news with 91% accuracy. The novelty of our approach consists in taking into account a series of characteristics related to users behavior on online social media, making a first, important step towards the smoothing of polarization and the mitigation of misinformation phenomena.

Motivation & Objective

• Motivation to mitigate misinformation by leveraging user polarization and confirmation bias.
• Develop a general, platform-agnostic framework to identify polarizing topics before they become fake-news targets.
• Extract topics and sentiment, derive behavior-based features, and apply machine learning to classify potential misinformation targets.
• Demonstrate the framework on a large Italian Facebook dataset and show its utility as a feature to improve fake-news detection.

Proposed method

• Four-phase framework: data collection from official and fake sources, topic extraction and sentiment analysis, feature definition, and classification.
• Entity-level sentiment and topic features including presentation distance, response distance, controversy, perception, and captivation.
• Supervised classification using multiple algorithms (Linear Regression, Logistic Regression, SVM, KNN, Neural Networks, Decision Trees) with data balancing.
• Threshold-based feature construction to separate controversial vs. non-controversial entities and identify potential misinformation targets.
• Evaluation using accuracy, precision, recall, F1, FP rate, and AUC to compare classifiers and select the best-performing models.

Experimental results

Research questions

• RQ1 Can we promptly identify polarizing topics on social media that are susceptible to misinformation?
• RQ2 How effective are polarization-based features in predicting future misinformation targets?
• RQ3 Can the polarization-derived signals serve as a useful feature to improve fake-news detection accuracy?
• RQ4 What are the key behavioral features that differentiate topics likely to be misinformation targets from others?

Key findings

• The framework identifies polarizing topics with 77% accuracy (0.73 AUC).
• When incorporated as features, the approach improves fake-news detection to 91% accuracy (0.94 AUC).
• Presentation distance and response distance are among the strongest features for distinguishing disputed topics.
• Threshold-based features (controlling for controversy, perception, and captivation) help separate controversial vs. non-controversial topics.
• Analyses on a large Italian Facebook dataset show that about 24 hours elapse between a topic’s first appearance in official news and its first appearance in fake news, with many topics propagating across both domains.