[Paper Review] Graph-based Modeling of Online Communities for Fake News Detection

Shantanu Chandra, Pushkar Mishra+4 more|arXiv (Cornell University)|Aug 14, 2020

Misinformation and Its Impacts56 references39 citations

TL;DR

SAFER builds a heterogeneous graph of users and articles and uses relational and hyperbolic graph neural networks to incorporate social-context features for improved fake news detection, achieving state-of-the-art results on two domains.

ABSTRACT

Over the past few years, there has been a substantial effort towards automated detection of fake news on social media platforms. Existing research has modeled the structure, style, content, and patterns in dissemination of online posts, as well as the demographic traits of users who interact with them. However, no attention has been directed towards modeling the properties of online communities that interact with the posts. In this work, we propose a novel social context-aware fake news detection framework, SAFER, based on graph neural networks (GNNs). The proposed framework aggregates information with respect to: 1) the nature of the content disseminated, 2) content-sharing behavior of users, and 3) the social network of those users. We furthermore perform a systematic comparison of several GNN models for this task and introduce novel methods based on relational and hyperbolic GNNs, which have not been previously used for user or community modeling within NLP. We empirically demonstrate that our framework yields significant improvements over existing text-based techniques and achieves state-of-the-art results on fake news datasets from two different domains.

Motivation & Objective

• Motivate exploiting online communities’ properties for fake news detection beyond text and simple propagation signals.
• Propose SAFER, a graph-based framework that aggregates article content, user sharing behavior, and social network structure.
• Systematically compare multiple GNN architectures, including relational and hyperbolic variants, for heterogeneous social graphs.
• Demonstrate improved detection performance over text-only baselines and existing methods on two domain-diverse datasets.

Proposed method

• Construct a two-type node heterogeneous graph with article nodes and user nodes where edges connect users to articles they shared and follow relationships between users.
• Encode article text with CNN or RoBERTa as a text encoder.
• Encode the user/communication graph with various GNNs (GCN, GAT, GraphSAGE, Relational GCN/Relational GAT, Hyperbolic GCN/GAT).
• Train encoders independently; aggregate per-article user embeddings into a social-context vector via normalized mean; concatenate with text embeddings; classify with logistic regression.
• Use Relational GNNs to capture different relation types in the graph; explore Hyperbolic GNNs to better embed scale-free, hierarchical structures.
• Evaluate on two datasets (GossipCop and HealthStory) with 70/10/20 train/valid/test splits; compare against text baselines, a social baseline, and prior methods.

Experimental results

Research questions

• RQ1 Can modeling online communities and user-content sharing relations improve fake news detection beyond text-based approaches?
• RQ2 Do relational and hyperbolic GNNs provide gains over traditional GNNs for heterogeneous social graphs in this task?
• RQ3 How does social-context information interact with textual content to affect detection accuracy across domains?
• RQ4 What is the impact of graph sparsity and active user selection on SAFER’s performance?
• RQ5 Is SAFER robust to potential biases from frequent users or biased sharing patterns?

Key findings

• SAFER significantly outperforms text-based baselines and majority-sharing baselines on GossipCop and HealthStory.
• Relational GNN variants outperform traditional GNNs, highlighting the value of explicitly modeling relations in heterogeneous graphs.
• Hyperbolic GNNs perform on par with traditional GNNs in the reported shallow (2-layer) setup but show potential advantages on deeper graphs; richer community modeling yields clearer class separation (visualized with t-SNE).
• Performance is sensitive to graph density and active user selection; SAFER remains strong under moderate sparsity and shows reduced gains as the graph becomes extremely sparse.
• GossipCop shows stronger social-context gains due to a larger share of type-(b) users (share-only-fake) and more distinct community structure than HealthStory.
• Ablation studies indicate optimal performance with top ~20K active users (HealthStory) and ~30K (GossipCop); overly dense or sparse graphs degrade performance.