[Paper Review] A Tensor Spectral Approach to Learning Mixed Membership Community Models

Animashree Anandkumar, Rong Ge+2 more|arXiv (Cornell University)|Feb 12, 2013

Tensor decomposition and applicationsMathematics30 references109 citations

TL;DR

This paper proposes a tensor spectral method for guaranteed community detection in mixed membership community models, where nodes can belong to multiple communities with fractional memberships. By leveraging low-order 3-star count tensors and spectral decomposition via SVD and power iterations, the approach achieves provable recovery of community memberships and model parameters with finite-sample guarantees, matching optimal scaling for the stochastic block model.

ABSTRACT

Detecting hidden communities from observed interactions is a classical problem. Theoretical analysis of community detection has so far been mostly limited to models with non-overlapping communities such as the stochastic block model. In this paper, we provide guaranteed community detection for a family of probabilistic network models with overlapping communities, termed as the mixed membership Dirichlet model, first introduced in Airoldi et al. (2008). This model allows for nodes to have fractional memberships in multiple communities and assumes that the community memberships are drawn from a Dirichlet distribution. Moreover, it contains the stochastic block model as a special case. We propose a unified approach to learning communities in these models via a tensor spectral decomposition approach. Our estimator uses low-order moment tensor of the observed network, consisting of 3-star counts. Our learning method is based on simple linear algebraic operations such as singular value decomposition and tensor power iterations. We provide guaranteed recovery of community memberships and model parameters, and present a careful finite sample analysis of our learning method. Additionally, our results match the best known scaling requirements for the special case of the (homogeneous) stochastic block model.

Motivation & Objective

• To address the limitation of existing community detection methods that assume non-overlapping communities.
• To provide a theoretically guaranteed method for learning communities in overlapping network models with mixed memberships.
• To extend theoretical guarantees from non-overlapping models like the stochastic block model to the more general mixed membership Dirichlet model.
• To develop a computationally efficient learning algorithm based on low-order moment tensors.

Proposed method

• The method uses the 3-star count tensor as a sufficient statistic of the network's higher-order structure.
• It applies tensor spectral decomposition via power iteration to extract latent community structure from the observed tensor.
• The approach leverages singular value decomposition (SVD) on unfolding matrices of the tensor to estimate community membership vectors.
• It models node memberships as draws from a Dirichlet distribution, enabling probabilistic inference over overlapping communities.
• The algorithm operates using only low-order moments, avoiding complex optimization or sampling procedures.
• The method is designed to be scalable and amenable to finite-sample analysis.

Experimental results

Research questions

• RQ1 Can community detection be reliably performed in overlapping network models with mixed memberships under theoretical guarantees?
• RQ2 How can low-order moment tensors be used to recover community structure in probabilistic network models?
• RQ3 What is the finite-sample performance of tensor-based methods in mixed membership models compared to existing approaches?
• RQ4 Does the proposed method achieve optimal scaling for the special case of the stochastic block model?
• RQ5 Can spectral techniques on tensors provide consistent estimation of both community memberships and model parameters?

Key findings

• The proposed tensor spectral method achieves guaranteed recovery of community memberships and model parameters under the mixed membership Dirichlet model.
• The method provides finite-sample error bounds that scale favorably with network size and community structure.
• The approach matches the best-known sample complexity scaling for the homogeneous stochastic block model, confirming its optimality in this special case.
• The use of 3-star count tensors enables consistent estimation with minimal higher-order moment computation.
• The algorithm is robust and computationally efficient, relying only on SVD and power iterations.
• The theoretical framework supports both overlapping and non-overlapping community structures, unifying prior approaches.