[Paper Review] Multi-way spectral partitioning and higher-order Cheeger inequalities

James R. Lee, Shayan Oveis Gharan+1 more|arXiv (Cornell University)|Nov 4, 2011

Graph theory and applications6 citations

TL;DR

This paper resolves a long-standing conjecture by proving that the multiplicity of the k-th smallest eigenvalue of the normalized Laplacian corresponds to the existence of k disjoint sparse cuts in a graph. It introduces an algorithmic multi-way spectral partitioning method using orthonormal eigenvectors, random projection, and geometric random partitioning, achieving a nearly optimal bound of $\rho_G(k) \leq O(\sqrt{\lambda_k \log k})$, with tightness shown on noisy hypercube graphs.

ABSTRACT

A basic fact in spectral graph theory is that the number of connected components in an undirected graph is equal to the multiplicity of the eigenvalue zero in the Laplacian matrix of the graph. In particular, the graph is disconnected if and only if there are at least two eigenvalues equal to zero. Cheeger's inequality and its variants provide an approximate version of the latter fact; they state that a graph has a sparse cut if and only if there are at least two eigenvalues that are close to zero. It has been conjectured that an analogous characterization holds for higher multiplicities, i.e., there are $k$ eigenvalues close to zero if and only if the vertex set can be partitioned into $k$ subsets, each defining a sparse cut. We resolve this conjecture. Our result provides a theoretical justification for clustering algorithms that use the bottom $k$ eigenvectors to embed the vertices into $\mathbb R^k$, and then apply geometric considerations to the embedding. We also show that these techniques yield a nearly optimal tradeoff between the expansion of sets of size $\approx n/k$, and the $k$th smallest eigenvalue of the normalized Laplacian matrix, denoted $λ_k$. In particular, we show that in every graph there is a set of size at most $2n/k$ which has expansion at most $O(\sqrt{λ_k \log k})$. This bound is tight, up to constant factors, for the "noisy hypercube" graphs.

Motivation & Objective

• To resolve the conjecture that $k$ eigenvalues close to zero imply the existence of $k$ disjoint sparse cuts in a graph.
• To provide a theoretical foundation for clustering algorithms that embed vertices into $\mathbb{R}^k$ using the bottom $k$ eigenvectors of the Laplacian.
• To establish a nearly optimal quantitative relationship between the $k$-th eigenvalue $\lambda_k$ and the $k$-way expansion constant $\rho_G(k)$.
• To develop an algorithmic framework for $k$-way spectral partitioning with provable guarantees on expansion quality.

Proposed method

• Construct $2k$ orthonormal eigenvectors $g_1, \dots, g_{2k}$ with small Rayleigh quotient, bounded by $O(\lambda_{2k})$.
• Apply spectral embedding $F: V \to \mathbb{R}^{2k}$ via $f_i(v) = g_i(v)/\sqrt{w(v)}$.
• Perform random projection into $\mathbb{R}^h$ with $h = O(\log k)$ using i.i.d. Gaussian vectors to reduce dimension.
• Apply random geometric partitioning by assigning each vertex to the nearest random center in a ball-based process in the projected space.
• Merge the resulting $m$ sets into $k' = \lceil 3k/2 \rceil$ sets by minimizing a mass-weighted norm $\mathcal{M}(S)$ to balance cluster quality.
• Apply a Cheeger sweep on each merged set to select $k$ sets with minimal expansion, using thresholding on $\|F^*(v)\|^2$.

Experimental results

Research questions

• RQ1 Does the existence of $k$ eigenvalues close to zero imply the existence of $k$ disjoint sparse cuts in a graph?
• RQ2 Can a provably good $k$-way partitioning be achieved using only the bottom $k$ eigenvectors of the Laplacian?
• RQ3 What is the tightest possible quantitative dependence of $\rho_G(k)$ on $\lambda_k$?
• RQ4 Is the $\sqrt{\log k}$ factor in the bound tight for the $k$-way expansion problem?
• RQ5 Can random projection and geometric partitioning be used to replace $k$-means in spectral clustering with theoretical guarantees?

Key findings

• The paper proves the conjecture that $\lambda_k$ being small implies the existence of $k$ disjoint sets, each with expansion $O(\sqrt{\lambda_k \log k})$.
• It establishes a nearly optimal upper bound: $\rho_G(k) \leq O(\sqrt{\lambda_k \log k})$, which is tight up to constant factors for the noisy hypercube graphs.
• The algorithm guarantees at least $k/2$ disjoint sets of expansion at most $O(\sqrt{\lambda_k \log k})$, one of which has size at most $2n/k$.
• The method achieves a $k$-way partitioning with each set having expansion $O(k^3 \sqrt{\lambda_k})$, improving upon prior bounds with explicit $k$-dependence.
• The proof is algorithmic and yields a near-linear time algorithm for $k$-way spectral partitioning using fast Laplacian solvers and random projection.
• The $\sqrt{\log k}$ factor is shown to be necessary and tight, as it cannot be improved beyond logarithmic factors in general graphs.