[Paper Review] Graph Neural Networks with Continual Learning for Fake News Detection from Social Media

Yi Han, Shanika Karunasekera+1 more|arXiv (Cornell University)|Jul 7, 2020

Misinformation and Its Impacts60 references77 citations

TL;DR

The paper uses propagation-based graph neural networks to detect fake news without text content and introduces continual learning to maintain performance on new, unseen data without full re-training.

ABSTRACT

Although significant effort has been applied to fact-checking, the prevalence of fake news over social media, which has profound impact on justice, public trust and our society, remains a serious problem. In this work, we focus on propagation-based fake news detection, as recent studies have demonstrated that fake news and real news spread differently online. Specifically, considering the capability of graph neural networks (GNNs) in dealing with non-Euclidean data, we use GNNs to differentiate between the propagation patterns of fake and real news on social media. In particular, we concentrate on two questions: (1) Without relying on any text information, e.g., tweet content, replies and user descriptions, how accurately can GNNs identify fake news? Machine learning models are known to be vulnerable to adversarial attacks, and avoiding the dependence on text-based features can make the model less susceptible to the manipulation of advanced fake news fabricators. (2) How to deal with new, unseen data? In other words, how does a GNN trained on a given dataset perform on a new and potentially vastly different dataset? If it achieves unsatisfactory performance, how do we solve the problem without re-training the model on the entire data from scratch? We study the above questions on two datasets with thousands of labelled news items, and our results show that: (1) GNNs can achieve comparable or superior performance without any text information to state-of-the-art methods. (2) GNNs trained on a given dataset may perform poorly on new, unseen data, and direct incremental training cannot solve the problem---this issue has not been addressed in the previous work that applies GNNs for fake news detection. In order to solve the problem, we propose a method that achieves balanced performance on both existing and new datasets, by using techniques from continual learning to train GNNs incrementally.

Motivation & Objective

• Motivate propagation-based fake news detection using non-textual social context features.
• Assess how GNNs distinguish fake vs real news from propagation patterns without tweet content.
• Investigate model performance on unseen datasets and the limitations of naive incremental training.
• Propose continual learning techniques to achieve balanced performance across existing and new data.

Proposed method

• Model propagation patterns of news as graphs where nodes are tweets/users and an extra node represents the news item.
• Use DiffPool, a graph classification GNN, built on GraphSage to classify propagation graphs.
• Construct adjacency and feature matrices from non-textual features such as user profile attributes and timeline-derived metrics.
• Experiment with different feature sets (user profiles, timeline features, or both) and with/without follower/following edges.
• Evaluate on PolitiFact and GossipCop datasets using accuracy, precision, recall, and F1 across multiple random splits.
• Apply continual learning approaches (GEM and EWC) to mitigate catastrophic forgetting when learning from multiple datasets.

Experimental results

Research questions

• RQ1 Can GNNs identify fake news using only non-textual propagation patterns, without tweet content?
• RQ2 How do GNNs trained on one dataset perform on a different, unseen dataset, and can continual learning improve cross-dataset generalization?

Key findings

• GNNs using non-textual propagation features achieve competitive or superior performance compared to text-based state-of-the-art methods on the evaluated datasets.
• Models trained on a single dataset perform poorly on another dataset, and naive incremental training fails to balance performance across datasets.
• Incorporating continual learning techniques (GEM, EWC) yields more balanced performance across datasets, with GEM generally outperforming EWC in experiments.
• Adding follower/following relations shows no significant improvement in performance for the considered settings.
• Training converges rapidly, with most models reaching stable performance within a modest number of epochs.