[Paper Review] Stochastic Block Models and Reconstruction

Elchanan Mossel, Joe Neeman+1 more|arXiv (Cornell University)|Feb 7, 2012

Topological and Geometric Data Analysis37 references142 citations

TL;DR

This paper rigorously proves that clustering in the sparse stochastic blockmodel is impossible when $(a-b)^2 < 2(a+b)$, confirming half of a conjecture by Decelle et al. based on statistical physics. It establishes a connection between clustering, spin-glass models, and reconstruction on the Bethe lattice, and shows that parameter estimation is also impossible in this regime, while providing an efficient algorithm for estimation when $(a-b)^2 > 2(a+b)$.

ABSTRACT

The planted partition model (also known as the stochastic blockmodel) is a classical cluster-exhibiting random graph model that has been extensively studied in statistics, physics, and computer science. In its simplest form, the planted partition model is a model for random graphs on $n$ nodes with two equal-sized clusters, with an between-class edge probability of $q$ and a within-class edge probability of $p$. Although most of the literature on this model has focused on the case of increasing degrees (ie.\ $pn, qn o \infty$ as $n o \infty$), the sparse case $p, q = O(1/n)$ is interesting both from a mathematical and an applied point of view. A striking conjecture of Decelle, Krzkala, Moore and Zdeborová based on deep, non-rigorous ideas from statistical physics gave a precise prediction for the algorithmic threshold of clustering in the sparse planted partition model. In particular, if $p = a/n$ and $q = b/n$, then Decelle et al.\ conjectured that it is possible to cluster in a way correlated with the true partition if $(a - b)^2 &gt; 2(a + b)$, and impossible if $(a - b)^2 &lt; 2(a + b)$. By comparison, the best-known rigorous result is that of Coja-Oghlan, who showed that clustering is possible if $(a - b)^2 &gt; C (a + b)$ for some sufficiently large $C$. We prove half of their prediction, showing that it is indeed impossible to cluster if $(a - b)^2 &lt; 2(a + b)$. Furthermore we show that it is impossible even to estimate the model parameters from the graph when $(a - b)^2 &lt; 2(a + b)$; on the other hand, we provide a simple and efficient algorithm for estimating $a$ and $b$ when $(a - b)^2 &gt; 2(a + b)$. Following Decelle et al, our work establishes a rigorous connection between the clustering problem, spin-glass models on the Bethe lattice and the so called reconstruction problem. This connection points to fascinating applications and open problems.

Motivation & Objective

• To resolve the algorithmic threshold for clustering in the sparse planted partition model, where edge probabilities scale as $a/n$ and $b/n$.
• To rigorously validate the non-rigorous statistical physics conjecture by Decelle et al. that clustering is possible if and only if $(a-b)^2 > 2(a+b)$.
• To establish a connection between clustering, reconstruction on the Bethe lattice, and spin-glass models in the sparse regime.
• To determine whether parameter estimation of $a$ and $b$ is possible from the graph structure under different conditions.
• To provide a simple, efficient algorithm for estimating $a$ and $b$ when $(a-b)^2 > 2(a+b)$.

Proposed method

• Uses a reconstruction problem framework on the Bethe lattice to analyze the information flow from the graph's structure to the latent community labels.
• Constructs a function $f$ on vertices based on their distance from a root, with values tied to the sign of labels at increasing depths, to model belief propagation.
• Analyzes the spectral properties of the adjacency matrix $A$ by comparing $Af$ to a scaled version of $f$, showing that $\|Af - \lambda f\|_2$ becomes negligible relative to $\|f\|_2$ when $|\theta| > (d-1)^{-1/2}$.
• Employs a recursive decomposition of sums over vertices at distance $r$ from a root to compute the variance of $Af(v) - \lambda f(v)$, showing it decays exponentially with depth $r$.
• Applies concentration inequalities to show that $\|f\|_2^2$ grows exponentially with depth, while $\|Af - \lambda f\|_2^2$ grows slower, implying $f$ is nearly an eigenvector of $A$.
• Uses the fact that if $f$ is close to an eigenvector, then the leading eigenvector of $A$ correlates with the true community labels, implying reconstruction is possible.

Experimental results

Research questions

• RQ1 Is clustering in the sparse stochastic blockmodel possible when $(a-b)^2 < 2(a+b)$?
• RQ2 Can the model parameters $a$ and $b$ be estimated from the graph when $(a-b)^2 < 2(a+b)$?
• RQ3 Does the reconstruction threshold $(a-b)^2 = 2(a+b)$ correspond to a phase transition in the information-theoretic limit?
• RQ4 Can spectral or belief propagation algorithms achieve clustering when $(a-b)^2 > 2(a+b)$?
• RQ5 What is the precise connection between the clustering problem, spin-glass models on the Bethe lattice, and the reconstruction problem?

Key findings

• It is impossible to cluster in a way correlated with the true partition when $(a-b)^2 < 2(a+b)$, confirming the impossibility half of the Decelle et al. conjecture.
• It is also impossible to estimate the parameters $a$ and $b$ from the graph when $(a-b)^2 < 2(a+b)$, even approximately.
• When $(a-b)^2 > 2(a+b)$, a simple and efficient algorithm exists for estimating $a$ and $b$ from the graph.
• The spectral properties of the adjacency matrix $A$ show that the leading eigenvector correlates with the true community structure when $(a-b)^2 > 2(a+b)$.
• The analysis confirms a deep connection between the clustering problem and the reconstruction problem on the Bethe lattice, linking it to spin-glass models in statistical physics.
• The threshold $(a-b)^2 = 2(a+b)$ marks a sharp phase transition in the information-theoretic feasibility of clustering and parameter estimation.