[Paper Review] A simpler spectral approach for clustering in directed networks

Simon Coste, Ludovic Stephan|arXiv (Cornell University)|Feb 5, 2021

Complex Network Analysis Techniques28 references4 citations

TL;DR

This paper proposes a simplified spectral clustering method for directed networks by directly using eigenvalues and eigenvectors of the unregularized adjacency matrix, bypassing SVD and symmetrization. It establishes theoretical guarantees via a Master Theorem for sparse, non-symmetric matrices and demonstrates superior performance in sparse regimes, with Gaussian Mixture clustering outperforming k-means on eigenvector embeddings.

ABSTRACT

We study the task of clustering in directed networks. We show that using the eigenvalue/eigenvector decomposition of the adjacency matrix is simpler than all common methods which are based on a combination of data regularization and SVD truncation, and works well down to the very sparse regime where the edge density has constant order. Our analysis is based on a Master Theorem describing sharp asymptotics for isolated eigenvalues/eigenvectors of sparse, non-symmetric matrices with independent entries. We also describe the limiting distribution of the entries of these eigenvectors; in the task of digraph clustering with spectral embeddings, we provide numerical evidence for the superiority of Gaussian Mixture clustering over the widely used k-means algorithm.

Motivation & Objective

• To challenge the conventional wisdom that directed network clustering requires symmetrization or SVD by showing eigen-decomposition of the adjacency matrix is sufficient.
• To provide a rigorous theoretical foundation for eigen-decomposition in sparse, non-symmetric directed networks using a Master Theorem for isolated eigenvalues and eigenvectors.
• To demonstrate empirically that Gaussian Mixture clustering on eigenvector embeddings outperforms k-means in directed network clustering.
• To extend spectral clustering theory beyond symmetric and dense networks to the sparse, directed case with inhomogeneous edge probabilities.

Proposed method

• Proposes direct eigen-decomposition of the directed adjacency matrix as the spectral embedding step, avoiding regularization or symmetrization.
• Develops a Master Theorem to describe sharp asymptotics of isolated eigenvalues and eigenvectors in sparse, non-symmetric random matrices with independent entries.
• Uses a branching process approximation (Galton-Watson tree) to model local neighborhood structure and derive limiting distributions of eigenvector entries.
• Applies Doob’s martingale convergence theorem and Slutsky’s lemma to prove convergence of pseudo-eigenvectors to limiting random variables on the tree.
• Translates convergence from the tree model to the finite graph using concentration inequalities and Portmanteau’s lemma for weak convergence.
• Employs normalized eigenvectors and shows their empirical distribution converges weakly to a non-degenerate limit distribution with explicit mean and variance.

Experimental results

Research questions

• RQ1 Can eigen-decomposition of the unregularized adjacency matrix outperform SVD-based or symmetrized methods in clustering directed networks?
• RQ2 What are the limiting distributions of eigenvector entries in sparse, non-symmetric directed networks?
• RQ3 How does the performance of spectral clustering based on adjacency matrix eigen-decomposition compare to SVD-based methods in the sparse regime?
• RQ4 Does Gaussian Mixture clustering provide better clustering accuracy than k-means when applied to eigenvector embeddings of directed networks?

Key findings

• The eigen-decomposition of the adjacency matrix yields consistent and accurate clustering results even in the very sparse regime where edge density is constant.
• The limiting distribution of eigenvector entries converges weakly to a non-degenerate random variable with explicit mean and variance, derived from a system of linear equations.
• Gaussian Mixture clustering applied to the eigenvector embeddings consistently outperforms k-means in terms of clustering accuracy on synthetic directed networks.
• Theoretical analysis confirms that the eigenvector components concentrate around their limiting distributions, with convergence in L² and in distribution.
• The method avoids the need for data regularization or symmetrization, simplifying the pipeline while maintaining or improving performance.