[Paper Review] Path-Based Spectral Clustering: Guarantees, Robustness to Outliers, and Fast Algorithms

Anna Little, Mauro Maggioni+1 more|arXiv (Cornell University)|Dec 17, 2017

Advanced Clustering Algorithms Research14 citations

TL;DR

This paper introduces path-based spectral clustering using the longest-leg path distance (LLPD) metric, which enables robust clustering of elongated and irregularly shaped clusters in high-dimensional data with significant outliers. It provides finite-sample guarantees for cluster recovery, proves the eigengap statistic correctly identifies the number of clusters, and introduces a quasilinear-time algorithm for efficient LLPD computation, enabling scalable spectral clustering on large datasets.

ABSTRACT

We consider the problem of clustering with the longest-leg path distance (LLPD) metric, which is informative for elongated and irregularly shaped clusters. We prove finite-sample guarantees on the performance of clustering with respect to this metric when random samples are drawn from multiple intrinsically low-dimensional clusters in high-dimensional space, in the presence of a large number of high-dimensional outliers. By combining these results with spectral clustering with respect to LLPD, we provide conditions under which the Laplacian eigengap statistic correctly determines the number of clusters for a large class of data sets, and prove guarantees on the labeling accuracy of the proposed algorithm. Our methods are quite general and provide performance guarantees for spectral clustering with any ultrametric. We also introduce an efficient, easy to implement approximation algorithm for the LLPD based on a multiscale analysis of adjacency graphs, which allows for the runtime of LLPD spectral clustering to be quasilinear in the number of data points.

Motivation & Objective

• To develop a clustering method robust to high-dimensional outliers and non-spherical cluster shapes.
• To provide theoretical finite-sample guarantees for LLPD-based spectral clustering under a flexible probabilistic data model.
• To prove that the eigengap statistic correctly identifies the number of clusters in the presence of noise and outliers.
• To design a fast, quasilinear-time algorithm for computing LLPD and its associated Laplacian eigensolver for scalability.
• To ensure high labeling accuracy in spectral clustering via perturbation bounds on eigenvectors.

Proposed method

• Uses the longest-leg path distance (LLPD) metric, defined as the minimum over all paths of the maximum edge length, to enhance separation between clusters.
• Employs a multiscale analysis of adjacency graphs to construct an efficient approximation algorithm for LLPD with quasilinear runtime.
• Applies spectral clustering on the LLPD-based similarity graph, leveraging the graph Laplacian to extract low-dimensional embeddings.
• Uses perturbation theory (Fan et al., 2018) to bound eigenvector deviation and ensure accurate spectral embedding.
• Introduces a rotation-invariant spectral embedding where cluster points are separated by at least √(2/N) in ℓ2 distance.
• Derives bounds on within-cluster LLPD and between-cluster LLPD using techniques from manifold learning and percolation theory.

Experimental results

Research questions

• RQ1 Can LLPD serve as a discriminatory metric that separates within-cluster and between-cluster distances with high probability in high-dimensional, outlier-contaminated data?
• RQ2 Under what conditions does the eigengap of the LLPD Laplacian correctly identify the true number of clusters?
• RQ3 How can LLPD be computed efficiently at scale, given its path-based nature?
• RQ4 What theoretical guarantees can be established for labeling accuracy in LLPD spectral clustering under non-parametric mixture models?
• RQ5 To what extent is the proposed method robust to geometric irregularities, varying cluster volumes, and high-dimensional noise?

Key findings

• The maximal within-cluster LLPD is bounded with high probability, while the minimal between-cluster LLPD is strictly larger, ensuring a clear phase transition.
• The eigengap statistic correctly identifies the number of clusters when the condition $\frac{1}{2} \geq 5\eta_1 + 4\eta_\theta + 6\zeta_N\eta_2 + O(\eta_1^2 + \eta_\theta^2 + \zeta_N^2\eta_2^2)$ holds.
• Perfect labeling accuracy is achieved when $\frac{1}{K^3\zeta_N^2} \gtrsim \eta_1 + \eta_\theta + \zeta_N\eta_2 + O(\eta_1^2 + \eta_\theta^2 + \zeta_N^2\eta_2^2)$, ensuring well-separated spectral embeddings.
• The proposed LLPD approximation algorithm runs in quasilinear time, making large-scale spectral clustering feasible.
• The spectral embedding error is bounded by $\|\tilde{\Phi}R - \Phi\|_{\text{max}} \leq P_3 = O\left(\frac{K^{5/2}\zeta_N^2}{\sqrt{N}}(\eta_1 + \eta_\theta + \zeta_N\eta_2)\right)$, ensuring stable clustering.
• The method is invariant to cluster geometry and robust to outliers, even when the number of outliers is large relative to inliers.