[Paper Review] Neural Diffusion Model for Microscopic Cascade Prediction

Cheng Yang, Maosong Sun+4 more|arXiv (Cornell University)|Dec 21, 2018

Complex Network Analysis Techniques43 references16 citations

TL;DR

This paper proposes a Neural Diffusion Model (NDM) for microscopic cascade prediction that relaxes strong assumptions of traditional models by leveraging attention mechanisms and convolutional networks to capture complex, user-level diffusion dynamics. The model achieves up to 26% relative improvement in F1 score over state-of-the-art baselines on four real-world datasets, demonstrating superior performance and generalization in early-stage prediction.

ABSTRACT

The prediction of information diffusion or cascade has attracted much attention over the last decade. Most cascade prediction works target on predicting cascade-level macroscopic properties such as the final size of a cascade. Existing microscopic cascade prediction models which focus on user-level modeling either make strong assumptions on how a user gets infected by a cascade or limit themselves to a specific scenario where "who infected whom" information is explicitly labeled. The strong assumptions oversimplify the complex diffusion mechanism and prevent these models from better fitting real-world cascade data. Also, the methods which focus on specific scenarios cannot be generalized to a general setting where the diffusion graph is unobserved. To overcome the drawbacks of previous works, we propose a Neural Diffusion Model (NDM) for general microscopic cascade prediction. NDM makes relaxed assumptions and employs deep learning techniques including attention mechanism and convolutional network for cascade modeling. Both advantages enable our model to go beyond the limitations of previous methods, better fit the diffusion data and generalize to unseen cascades. Experimental results on diffusion prediction task over four realistic cascade datasets show that our model can achieve a relative improvement up to 26% against the best performing baseline in terms of F1 score.

Motivation & Objective

• To address the limitations of existing microscopic cascade prediction models that rely on strong, oversimplified assumptions about user infection mechanisms.
• To enable generalization to real-world settings where the diffusion graph is unobserved, unlike prior methods requiring explicit 'who infected whom' information.
• To improve prediction accuracy and robustness in early-stage cascade forecasting using deep learning techniques.
• To model complex, non-Markovian user-level diffusion dynamics that cannot be captured by static probability models like Independent Cascade.
• To enhance interpretability by analyzing learned attention and convolutional projection matrices, linking structural patterns to prediction behavior.

Proposed method

• The model uses a multi-head attention mechanism to project user embeddings into 16 subspaces, each representing a distinct role or influence pattern of the user in the diffusion process.
• A 1D convolutional network processes the sequence of recently infected users, with learnable projection matrices for each position (e.g., t, t-1, t-2) to weigh their contributions to the next infection.
• The model incorporates an initial user embedding with a learnable scale, and the relative magnitude of projection matrices (measured by Frobenius norm) reveals the importance of each time step in prediction.
• The attention mechanism allows the model to dynamically attend to relevant user features and roles, improving representation learning without relying on hand-crafted features.
• The architecture is trained end-to-end on cascade sequences to predict the next infected user, with loss optimized via cross-entropy on the next-step prediction task.
• The model is designed to be generalizable, requiring no prior knowledge of the diffusion graph, making it applicable to scenarios like product adoption or viral spread where infection chains are unobserved.

Experimental results

Research questions

• RQ1 Can a deep learning-based model outperform traditional cascade models in microscopic cascade prediction when strong assumptions about infection probabilities are relaxed?
• RQ2 To what extent can attention mechanisms and convolutional networks model complex, non-Markovian user-level diffusion dynamics in real-world cascade data?
• RQ3 How do the relative contributions of recent infected users (e.g., most recent vs. second most recent) influence prediction accuracy, and can this be quantified via learned model parameters?
• RQ4 Does the initial user play a dominant role in predicting future infections, and does this vary across different data distributions such as Twitter vs. other networks?
• RQ5 Can the learned model parameters (e.g., projection matrix norms) provide interpretable insights into the underlying diffusion process?

Key findings

• The Neural Diffusion Model achieves a relative improvement of up to 26% in F1 score over the best baseline on the diffusion prediction task across four real-world datasets.
• The scale of the learned projection matrix for the most recent infected user (W₀ᶜ) is consistently larger than for older users (W₁ᶜ, W₂ᶜ), confirming that recent users contribute more to next-step predictions.
• On the Twitter dataset, the initial user embedding projection matrix (W_initᶜ) has the largest Frobenius norm, indicating its dominant role in the diffusion process, likely due to the complete history of URL propagation.
• The model generalizes well to unseen cascades and performs particularly strongly in early-stage prediction, where accurate forecasting is most valuable for real-world applications.
• The attention mechanism enables the model to learn meaningful, dynamic user role representations without relying on hand-crafted features, though full interpretability remains a challenge.
• The model’s performance is competitive or superior even when the initial user is not the true source, suggesting robustness to partial observation and data windowing effects.