[Paper Review] Understanding Regularized Spectral Clustering via Graph Conductance

Yilin Zhang, Karl Rohe|arXiv (Cornell University)|Jun 5, 2018

Complex Network Analysis TechniquesPhysics and Astronomy47 citations

TL;DR

The paper links graph conductance to spectral clustering to explain Vanilla-SC failures on sparse graphs and shows regularization via CoreCut improves balance and robustness, with faster computation.

ABSTRACT

This paper uses the relationship between graph conductance and spectral clustering to study (i) the failures of spectral clustering and (ii) the benefits of regularization. The explanation is simple. Sparse and stochastic graphs create a lot of small trees that are connected to the core of the graph by only one edge. Graph conductance is sensitive to these noisy `dangling sets'. Spectral clustering inherits this sensitivity. The second part of the paper starts from a previously proposed form of regularized spectral clustering and shows that it is related to the graph conductance on a `regularized graph'. We call the conductance on the regularized graph CoreCut. Based upon previous arguments that relate graph conductance to spectral clustering (e.g. Cheeger inequality), minimizing CoreCut relaxes to regularized spectral clustering. Simple inspection of CoreCut reveals why it is less sensitive to small cuts in the graph. Together, these results show that unbalanced partitions from spectral clustering can be understood as overfitting to noise in the periphery of a sparse and stochastic graph. Regularization fixes this overfitting. In addition to this statistical benefit, these results also demonstrate how regularization can improve the computational speed of spectral clustering. We provide simulations and data examples to illustrate these results.

Motivation & Objective

• Explain why Vanilla-SC fails on sparse and stochastic graphs due to peripheral dangling sets.
• Introduce CoreCut as a regularized graph conductance related to Regularized-SC.
• Show how Regularized-SC mitigates overfitting and improves partition balance.
• Demonstrate computational benefits of regularization in spectral clustering through experiments.

Proposed method

• Relate graph conductance to spectral clustering via Cheeger inequality to motivate regularization.
• Define g-dangling sets and show they yield small conductance in sparse graphs.
• Prove that real-world sparse graphs contain many g-dangling sets, causing many small eigenvalues and slow convergence.
• Introduce CoreCut as regularized conductance on a regularized graph G_tau and relate to Regularized-SC.
• Derive bounds showing when CoreCut favors core partitions over peripheral cuts under certain tau choices.
• Provide empirical simulations and real-data experiments comparing Vanilla-SC and Regularized-SC.

Experimental results

Research questions

• RQ1 How does graph conductance relate to failures of Vanilla-SC on sparse, stochastic graphs?
• RQ2 How does regularization alter the conductance landscape to favor core graph partitions?
• RQ3 What is CoreCut, and how does it connect to Regularized-SC through graph regularization?
• RQ4 Do Regularized-SC solutions improve partition balance and reduce overfitting in practice?
• RQ5 What are the computational implications of using regularized spectral clustering?

Key findings

• Sparse and stochastic graphs contain many g-dangling sets that produce small conductance values and many small eigenvalues, signaling noise rather than structure.
• CoreCut regularization shifts conductance to ignore small peripheral cuts and emphasize core graph structure, aligning with Regularized-SC.
• Regularized-SC yields more balanced partitions compared to Vanilla-SC, which tends to produce unbalanced, noise-driven cuts.
• Regularized-SC computations are faster for obtaining the second eigenvector than Vanilla-SC in the reported experiments.