[Paper Review] Local Algorithms for Block Models with Side Information

Elchanan Mossel, Jiaming Xu|arXiv (Cornell University)|Aug 10, 2015

Machine Learning and Algorithms42 references3 citations

TL;DR

This paper demonstrates that local algorithms, specifically belief propagation on local neighborhoods, achieve optimal performance in community detection for sparse stochastic block models when side information—correlated vertex labels—is available. Unlike in symmetric block models without side information, where local algorithms are suboptimal, the presence of side information enables local methods to maximize the expected fraction of correctly labeled vertices across three distinct parameter regimes.

ABSTRACT

There has been a recent interest in understanding the power of local algorithms for optimization and inference problems on sparse graphs. Gamarnik and Sudan (2014) showed that local algorithms are weaker than global algorithms for finding large independent sets in sparse random regular graphs. Montanari (2015) showed that local algorithms are suboptimal for finding a community with high connectivity in the sparse Erdős-Rényi random graphs. For the symmetric planted partition problem (also named community detection for the block models) on sparse graphs, a simple observation is that local algorithms cannot have non-trivial performance. In this work we consider the effect of side information on local algorithms for community detection under the binary symmetric stochastic block model. In the block model with side information each of the $n$ vertices is labeled $+$ or $-$ independently and uniformly at random; each pair of vertices is connected independently with probability $a/n$ if both of them have the same label or $b/n$ otherwise. The goal is to estimate the underlying vertex labeling given 1) the graph structure and 2) side information in the form of a vertex labeling positively correlated with the true one. Assuming that the ratio between in and out degree $a/b$ is $Θ(1)$ and the average degree $ (a+b) / 2 = n^{o(1)}$, we characterize three different regimes under which a local algorithm, namely, belief propagation run on the local neighborhoods, maximizes the expected fraction of vertices labeled correctly. Thus, in contrast to the case of symmetric block models without side information, we show that local algorithms can achieve optimal performance for the block model with side information.

Motivation & Objective

• To investigate whether local algorithms can achieve optimal performance in community detection when side information about vertex labels is available.
• To resolve the gap between local and global algorithms in sparse stochastic block models, where local methods typically fail without side information.
• To characterize the regimes in which belief propagation on local neighborhoods maximizes the fraction of correctly labeled vertices.
• To analyze the asymptotic performance of belief propagation using density evolution and Gaussian approximation techniques in the large-degree limit.

Proposed method

• Formalizes a binary symmetric stochastic block model with side information: each vertex is independently labeled + or −, and edges are generated with probability a/n (same label) or b/n (different labels).
• Applies belief propagation on local neighborhoods of radius o(log n) to estimate vertex labels using both graph structure and side information.
• Uses density evolution with Gaussian approximation to derive recursive equations for message variances and belief update dynamics.
• Analyzes the fixed points of the recursion to characterize the asymptotic expected fraction of correctly labeled vertices.
• Employs symmetry and moment-matching arguments to prove that the expected performance of belief propagation matches the theoretical optimum in specific regimes.
• Derives a closed-form expression for the limiting performance in the large-degree limit (a→∞) via the fixed point of the variance recursion.

Experimental results

Research questions

• RQ1 Can local algorithms achieve optimal performance in community detection when side information is present?
• RQ2 In which parameter regimes does belief propagation on local neighborhoods maximize the expected fraction of correctly labeled vertices?
• RQ3 How does the presence of side information alter the fundamental limitations of local algorithms in sparse block models?
• RQ4 What is the asymptotic performance of belief propagation in the large-degree limit, and how does it relate to the optimal achievable accuracy?
• RQ5 Under what conditions do the upper and lower bounds on performance converge, implying optimality of the local algorithm?

Key findings

• Belief propagation on local neighborhoods achieves optimal performance in the symmetric block model with side information when |a−b|<2 and all 0<α<1/2.
• Optimality is also achieved when (a−b)²>C(a+b) for some constant C and all 0<α<1/2.
• For all a,b, belief propagation is optimal if the label error probability is at most α∗∈(0,1/2), a constant.
• In the large-degree limit (a→∞), the expected fraction of correctly labeled vertices converges to 1−𝔼[Q((v∗+U)/√v∗)], where v∗ is the smallest fixed point of v=μ²h(v)/4.
• The performance bounds from belief propagation and the global optimum coincide when |μ|<2 or when α≤α∗, implying that the fixed point v∗ is unique and the algorithm is optimal.
• The variance recursion for belief propagation messages converges to a fixed point, and the difference between upper and lower bounds on performance vanishes in the limit, confirming optimality.