[Paper Review] Inferring Networks of Diffusion and Influence

Manuel Gomez-Rodriguez, Jure Leskovec+1 more|arXiv (Cornell University)|Jun 1, 2010

Complex Network Analysis Techniques236 citations

TL;DR

This paper proposes NetInf, a scalable algorithm that infers influence and diffusion networks from observed infection or adoption times, using submodular optimization to find a near-optimal network structure. It demonstrates that real-world news diffusion exhibits a core-periphery structure, with a few influential media sites connecting topic-specific clusters.

ABSTRACT

Information diffusion and virus propagation are fundamental processes taking place in networks. While it is often possible to directly observe when nodes become infected with a virus or adopt the information, observing individual transmissions (i.e., who infects whom, or who influences whom) is typically very difficult. Furthermore, in many applications, the underlying network over which the diffusions and propagations spread is actually unobserved. We tackle these challenges by developing a method for tracing paths of diffusion and influence through networks and inferring the networks over which contagions propagate. Given the times when nodes adopt pieces of information or become infected, we identify the optimal network that best explains the observed infection times. Since the optimization problem is NP-hard to solve exactly, we develop an efficient approximation algorithm that scales to large datasets and finds provably near-optimal networks. We demonstrate the effectiveness of our approach by tracing information diffusion in a set of 170 million blogs and news articles over a one year period to infer how information flows through the online media space. We find that the diffusion network of news for the top 1,000 media sites and blogs tends to have a core-periphery structure with a small set of core media sites that diffuse information to the rest of the Web. These sites tend to have stable circles of influence with more general news media sites acting as connectors between them.

Motivation & Objective

• To infer the underlying network over which information, influence, or viruses spread when only infection times are observed.
• To address the challenge of reconstructing unobserved propagation networks in large-scale systems like online media.
• To develop a scalable, provably near-optimal algorithm for network inference under partial observations.
• To uncover structural properties of real-world diffusion networks, such as core-periphery organization and influence clusters.
• To enable large-scale analysis of information flow in social and media networks using only temporal adoption data.

Proposed method

• Formulates the network inference problem as a maximum likelihood estimation over a generative model of cascades.
• Models diffusion as a stochastic process on an unknown directed network, where each node's infection time depends on its neighbors.
• Reframes the problem as selecting the optimal set of k directed edges to maximize likelihood, proven to be NP-hard.
• Leverages the submodularity of the likelihood function to design a greedy approximation algorithm with performance guarantees.
• Employs localized updates and lazy evaluation to scale efficiently to large datasets with millions of nodes and cascades.
• Uses a heuristic baseline for comparison, demonstrating NetInf's superior accuracy and scalability.

Experimental results

Research questions

• RQ1 What is the underlying network structure over which information spreads in online media, given only infection times?
• RQ2 How can we infer the most likely network of influence from partial observations of adoption times?
• RQ3 What global structural properties (e.g., core-periphery, clustering) emerge in real-world diffusion networks?
• RQ4 How does NetInf compare to heuristic baselines in reconstructing true networks from limited data?
• RQ5 Can the inferred network reveal roles of individual media sites, such as core influencers or connectors between communities?

Key findings

• NetInf accurately recovers the true underlying network from synthetic data using only a small number of cascades.
• On a real dataset of 170 million blog and news articles, NetInf inferred a diffusion network with a clear core-periphery structure.
• The top 1,000 media sites and blogs form a core network that diffuses information to the broader blogosphere.
• Media sites with stable circles of influence act as hubs, while general news outlets serve as connectors between topic-specific clusters.
• The inferred network reveals distinct topic-based clusters (e.g., politics, technology, gossip) linked by a few influential core sites.
• NetInf significantly outperforms a maximum-weight heuristic baseline in both accuracy and scalability on synthetic and real data.