[Paper Review] RP-DNN: A Tweet level propagation context based deep neural networks for early rumor detection in Social Media

Jie Gao, Sooji Han+2 more|arXiv (Cornell University)|Feb 28, 2020

Misinformation and Its Impacts52 references22 citations

TL;DR

This paper proposes RP-DNN, a hybrid deep neural network that combines character-level bidirectional language modeling and stacked LSTMs to detect rumors at the tweet level by modeling textual content and social-temporal propagation contexts (replies, retweets, metadata). It achieves state-of-the-art performance in early rumor detection on a large, multi-event dataset using leave-one-out cross-validation, outperforming existing methods in detecting unseen rumors before widespread dissemination.

ABSTRACT

Early rumor detection (ERD) on social media platform is very challenging when limited, incomplete and noisy information is available. Most of the existing methods have largely worked on event-level detection that requires the collection of posts relevant to a specific event and relied only on user-generated content. They are not appropriate to detect rumor sources in the very early stages, before an event unfolds and becomes widespread. In this paper, we address the task of ERD at the message level. We present a novel hybrid neural network architecture, which combines a task-specific character-based bidirectional language model and stacked Long Short-Term Memory (LSTM) networks to represent textual contents and social-temporal contexts of input source tweets, for modelling propagation patterns of rumors in the early stages of their development. We apply multi-layered attention models to jointly learn attentive context embeddings over multiple context inputs. Our experiments employ a stringent leave-one-out cross-validation (LOO-CV) evaluation setup on seven publicly available real-life rumor event data sets. Our models achieve state-of-the-art(SoA) performance for detecting unseen rumors on large augmented data which covers more than 12 events and 2,967 rumors. An ablation study is conducted to understand the relative contribution of each component of our proposed model.

Motivation & Objective

• To address the challenge of early rumor detection (ERD) when only limited, noisy, and incomplete information is available in the initial stages of rumor spread.
• To develop a tweet-level ERD system that leverages not only textual content but also social-temporal propagation patterns such as replies and retweets.
• To create a context-aware deep learning model that integrates multiple correlated inputs—source content, conversational content, and metadata—into a unified, noise-resilient representation.
• To improve generalizability and robustness by training on a large, augmented dataset covering over 12 real-world rumor events and 2,967 rumors.
• To demonstrate the effectiveness of multi-layered attention mechanisms in jointly learning contextual representations from diverse input modalities for rumor detection.

Proposed method

• The model uses a task-specific character-based bidirectional language model to encode the textual content of source tweets, capturing fine-grained linguistic patterns.
• It employs stacked Long Short-Term Memory (LSTM) networks to model the sequential dynamics of social context, including reply and retweet chains.
• Social context is decomposed into two components: conversational content (CC), representing the flow of discussion, and social context metadata (CM), including temporal, structural, and user-level signals.
• A multi-layered attention mechanism is applied to jointly attend over multiple context inputs (SC, CC, CM), enabling the model to focus on relevant propagation patterns and suppress noise.
• The framework integrates textual and social-temporal representations into a unified context-aware embedding space, enhancing robustness to incomplete or noisy data.
• The model is trained and evaluated using a stringent leave-one-out cross-validation (LOO-CV) setup across seven public rumor datasets, with extensive ablation studies to validate component contributions.

Experimental results

Research questions

• RQ1 Can a deep learning model that integrates both textual content and social-temporal propagation context improve early rumor detection performance at the tweet level?
• RQ2 How effective is a task-specific character-level bidirectional language model in capturing subtle linguistic cues indicative of rumors compared to word-level models?
• RQ3 To what extent do multi-layered attention mechanisms enhance the model’s ability to jointly learn from heterogeneous inputs such as replies, retweets, and metadata?
• RQ4 Does incorporating social context metadata (e.g., timing, user activity) improve detection accuracy in the early stages of rumor propagation?
• RQ5 How generalizable is the proposed model across diverse, unseen rumor events, especially when training data is limited?

Key findings

• RP-DNN achieves state-of-the-art performance in tweet-level early rumor detection, outperforming existing SoA models on a large, augmented dataset covering more than 12 real-world rumor events and 2,967 rumors.
• The model significantly improves detection accuracy on unseen rumors, demonstrating strong generalization under a stringent leave-one-out cross-validation setup.
• Ablation studies confirm that both the character-level language model and the multi-layered attention mechanism are critical for performance, with each component contributing meaningfully to the final result.
• The integration of social context metadata (CM) and conversational content (CC) leads to a measurable improvement in detection robustness, especially in noisy or sparse early-stage scenarios.
• The model achieves comparable performance to state-of-the-art event-level rumor detection systems, despite operating at the finer-grained tweet level, highlighting its effectiveness in early detection.
• The use of a task-specific, fine-tuned language model for rumor detection yields better performance than generic pre-trained models, underscoring the value of domain-specific adaptation.