[Paper Review] Spectral clustering under degree heterogeneity: a case for the random walk Laplacian

Alexander Modell, Patrick Rubin‐Delanchy|arXiv (Cornell University)|May 3, 2021

Bayesian Methods and Mixture Models4 citations

TL;DR

This paper advocates for using the random walk Laplacian in spectral clustering to inherently correct for degree heterogeneity in networks. It proves that the resulting embedding consistently estimates degree-corrected latent positions, enabling asymptotically perfect community recovery without spherical projection, and proposes a weighted Gaussian mixture model for optimal clustering.

ABSTRACT

This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. Under a generalised random dot product graph, the embedding provides uniformly consistent estimates of degree-corrected latent positions, with asymptotically Gaussian error. In the special case of a degree-corrected stochastic block model, the embedding concentrates about K distinct points, representing communities. These can be recovered perfectly, asymptotically, through a subsequent clustering step, without spherical projection, as commonly required by algorithms based on the adjacency or normalised, symmetric Laplacian matrices. While the estimand does not depend on degree, the asymptotic variance of its estimate does -- higher degree nodes are embedded more accurately than lower degree nodes. Our central limit theorem therefore suggests fitting a weighted Gaussian mixture model as the subsequent clustering step, for which we provide an expectation-maximisation algorithm.

Motivation & Objective

• To provide a principled statistical interpretation of spectral embeddings derived from the random walk Laplacian.
• To address the limitation of standard spectral clustering methods in handling degree heterogeneity, especially in models like the degree-corrected stochastic block model.
• To demonstrate that the random walk Laplacian produces embeddings that are inherently corrected for node degree, eliminating the need for spherical projection.
• To develop a theoretically grounded clustering procedure that accounts for the asymptotic variance dependence on node degree.
• To show that perfect community recovery is achievable asymptotically using the proposed embedding and a weighted Gaussian mixture model.

Proposed method

• Uses the random walk Laplacian matrix as the input to spectral embedding, replacing the adjacency or symmetric Laplacian matrices.
• Analyzes the embedding under a generalized random dot product graph model, where node latent positions are scaled by node-specific weights.
• Establishes uniform consistency of the embedding for degree-corrected latent positions under the model.
• Derives a central limit theorem showing that the estimation error is asymptotically Gaussian, with variance depending on node degree.
• Proposes a weighted Gaussian mixture model for clustering, where weights are inversely proportional to the asymptotic variance of each node’s embedding.
• Provides an expectation-maximization algorithm to fit the weighted Gaussian mixture model, improving clustering accuracy over standard methods.

Experimental results

Research questions

• RQ1 How does spectral embedding using the random walk Laplacian compare to embeddings from the adjacency or symmetric Laplacian matrices in terms of degree correction?
• RQ2 Can the random walk Laplacian produce consistent estimates of degree-corrected latent positions in a generalized random dot product graph?
• RQ3 Does the resulting embedding allow for asymptotically perfect community recovery in the degree-corrected stochastic block model without spherical projection?
• RQ4 How does the asymptotic variance of the embedding estimates depend on node degree, and can this be leveraged to improve clustering?
• RQ5 Is a weighted Gaussian mixture model superior to standard clustering methods for recovering communities from embeddings of the random walk Laplacian?

Key findings

• Spectral embedding using the random walk Laplacian produces uniformly consistent estimates of degree-corrected latent positions in a generalized random dot product graph.
• The asymptotic distribution of the embedding error is Gaussian, with variance that increases with lower node degree, meaning higher-degree nodes are embedded more accurately.
• In the degree-corrected stochastic block model, the embedding concentrates around K distinct points corresponding to communities, enabling perfect asymptotic recovery without spherical projection.
• The proposed method outperforms spherical LSE and is competitive with spherical ASE, especially for large networks and in dense regimes.
• The use of a weighted Gaussian mixture model, with weights derived from the asymptotic variance, leads to improved clustering performance over unweighted alternatives.
• Empirical results on the Harry Potter enmity network show clear community separation in the one-dimensional random walk embedding, with outliers like Snape and Black correctly positioned based on narrative role.