[Paper Review] A Comprehensive Low and High-level Feature Analysis for Early Rumor Detection on Twitter

Tu N. Nguyen|arXiv (Cornell University)|Nov 2, 2017

Misinformation and Its Impacts20 references3 citations

TL;DR

This paper proposes a cascaded rumor detection model that leverages deep neural networks to learn low-level, hidden representations from individual tweets at early stages, combining them with high-level features to improve early rumor detection. The method achieves 87% accuracy within 25 hours—matching human editors—by analyzing temporal dynamics of features over 48 hours, showing that low-level representations are among the most effective early signals.

ABSTRACT

Recent work have done a good job in modeling rumors and detecting them over microblog streams. However, the performance of their automatic approaches are not relatively high when looking early in the diffusion. A first intuition is that, at early stage, most of the aggregated rumor features (e.g., propagation features) are not mature and distinctive enough. The objective of rumor debunking in microblogs, however, are to detect these misinformation as early as possible. In this work, we leverage neural models in learning the hidden representations of individual rumor-related tweets at the very beginning of a rumor. Our extensive experiments show that the resulting signal improves our classification performance over time, significantly within the first 10 hours. To deepen the understanding of these low and high-level features in contributing to the model performance over time, we conduct an extensive study on a wide range of high impact rumor features for the 48 hours range. The end model that engages these features are shown to be competitive, reaches over 90% accuracy and out-performs strong baselines in our carefully cured dataset.

Motivation & Objective

• Address the challenge of early rumor detection when propagation-based features are underdeveloped.
• Improve rumor detection performance during the first 10–48 hours of a rumor's spread, when signals are sparse.
• Investigate the temporal evolution of low- and high-level features to understand their contribution to classification over time.
• Develop a cascaded model that integrates single-tweet credibility scores with time-series feature dynamics for improved detection.
• Benchmark the system against human experts to validate early detection capability and real-world relevance.

Proposed method

• Use a pre-trained Convolutional Neural Network (CNN) to learn deep, low-level representations from individual tweet texts, capturing hidden credibility signals beyond simple keywords.
• Compute a 'CreditScore' for each tweet based on its predicted credibility from the neural model, serving as a proxy for truthfulness at the tweet level.
• Extract a wide range of high-level features (e.g., propagation patterns, sentiment, URL content, user demographics) and structure them in a time-series format to model temporal dynamics.
• Combine CreditScores and high-level features into a dynamic feature vector that evolves over time, capturing the progression of an event.
• Train a classifier on this time-series feature representation to distinguish rumors from real news, with performance evaluated at multiple time points.
• Apply a threshold of 72 hours to align with human fact-checking timelines, using Snopes as a ground truth source for rumor confirmation times.

Experimental results

Research questions

• RQ1 How do low-level neural representations of individual tweets contribute to early rumor detection compared to traditional high-level features?
• RQ2 How do the predictive performances of different feature types (low-level vs. high-level) evolve over the first 48 hours of a rumor's spread?
• RQ3 Why do high-level features like propagation patterns underperform at early stages despite their effectiveness later?
• RQ4 Can a cascaded model combining single-tweet credibility with time-series feature dynamics outperform strong baselines in early rumor detection?
• RQ5 How does the system's performance compare to human fact-checkers in terms of detection speed and accuracy?

Key findings

• Low-level neural representations from individual tweets are among the top-performing features for early rumor detection, outperforming many high-level features in the first 10 hours.
• The model achieves 87% accuracy within 25 hours—matching the average time human editors take to verify rumors—demonstrating strong early detection capability.
• High-level features such as propagation patterns and user-based signals show poor performance at early stages due to their inherently accumulating nature, which requires time to mature.
• The CreditScore, derived from the neural credibility model, stabilizes over time and remains a strong signal, even when individual tweets are noisy or contradictory.
• The dynamic time-series feature model effectively captures temporal evolution of features, improving classification performance across all time windows up to 48 hours.
• The study reveals that while high-level features are powerful later in the rumor lifecycle, low-level representations are critical for early detection, suggesting a need for better integration of diverse signal sources in future models.