[Paper Review] Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Jean Barbier, Chun Lam Chan+1 more|arXiv (Cornell University)|Feb 19, 2019

Random Matrices and Applications30 references4 citations

TL;DR

This paper presents a direct, self-contained proof of a single-letter variational expression for the mutual information in the asymmetric two-groups stochastic block model using the adaptive interpolation method. Unlike prior indirect approaches that map the problem to matrix estimation, this work derives the result purely within the SBM framework, avoiding Gaussian integration by parts through a novel approximate integration by parts formula for Bernoulli variables.

ABSTRACT

We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping the model to a rank-one matrix estimation problem whose mutual information is then determined by a combination of methods (e.g., interpolation, cavity, algorithmic, spatial coupling). In this contribution we provide a self-contained direct method using only the recently introduced adaptive interpolation method.

Motivation & Objective

• To provide a rigorous, direct derivation of the asymptotic mutual information in the dense asymmetric two-groups stochastic block model.
• To overcome limitations of prior indirect proofs that rely on mapping to rank-one matrix estimation problems.
• To unify and simplify existing approaches by using only the adaptive interpolation method without detours to other models.
• To develop a new approximate integration by parts formula tailored for Bernoulli random variables in the adjacency matrix.
• To establish tight upper and lower bounds on mutual information using a single, consistent method.

Proposed method

• The adaptive interpolation method is applied directly to the stochastic block model, avoiding indirect mappings to matrix factorization problems.
• A novel approximate integration by parts formula is derived for Bernoulli-distributed adjacency matrix entries, replacing standard Gaussian integration by parts.
• The method constructs a continuous interpolation between the original model and a simpler reference model via a parameter $ t \in [0,1] $, tracking the derivative of the free energy.
• The proof uses a two-step interpolation: first introducing a perturbation parameter $ \epsilon $, then a time-like parameter $ t $, to control the interpolation path.
• The derivative of the free energy with respect to $ t $ is computed using the new approximate integration by parts, leading to a variational expression.
• The resulting expression is shown to converge to a single-letter variational formula in the large-$ n $ limit, yielding the mutual information.

Experimental results

Research questions

• RQ1 Can a direct, self-contained proof of the mutual information in the asymmetric two-groups stochastic block model be achieved without mapping to a matrix estimation problem?
• RQ2 What is the role of the adaptive interpolation method in deriving tight upper and lower bounds for mutual information in this context?
• RQ3 How can integration by parts be adapted for non-Gaussian, Bernoulli-distributed variables in the context of graphical models?
• RQ4 Does the adaptive interpolation method yield a unified framework that covers both symmetric and asymmetric community sizes?
• RQ5 Can the method avoid the technical complications of Gaussian integration by parts while maintaining rigor and tightness?

Key findings

• The paper establishes a single-letter variational expression for the asymptotic mutual information in the dense asymmetric two-groups stochastic block model.
• The result is derived directly from the SBM without mapping to a rank-one matrix factorization problem, providing a more transparent and unified proof.
• The adaptive interpolation method yields both tight upper and lower bounds, unlike classical interpolation which only gives one-sided inequalities.
• A new approximate integration by parts formula is developed specifically for Bernoulli variables in the adjacency matrix, enabling the derivation without Gaussian assumptions.
• The method avoids the use of message-passing or cavity methods, and does not rely on second-order phase transition assumptions.
• The proof is valid in the widest possible dense regime, including cases with first-order (discontinuous) phase transitions due to asymmetry.