[Paper Review] Graph Connectivity in Noisy Sparse Subspace Clustering

Yining Wang, Yu-Xiang Wang+1 more|arXiv (Cornell University)|Apr 4, 2015

Sparse and Compressive Sensing Techniques3 citations

TL;DR

This paper proposes a post-processing procedure for noisy sparse subspace clustering (SSC) that ensures exact clustering by merging subspaces based on angular distance, under general position or restricted eigenvalue assumptions. It provides the first exact clustering guarantee for noisy SSC with subspaces of dimension greater than 3, proven under a fully deterministic model with adversarial noise.

ABSTRACT

Subspace clustering is the problem of clustering data points into a union of low-dimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image processing and machine learning. A line of recent work (4, 19, 24, 20) provided strong theoretical guarantee for sparse subspace clustering (4), the state-of-the-art algorithm for subspace clustering, on both noiseless and noisy data sets. It was shown that under mild conditions, with high probability no two points from different subspaces are clustered together. Such guarantee, however, is not sufficient for the clustering to be correct, due to the notorious "graph connectivity problem" (15). In this paper, we investigate the graph connectivity problem for noisy sparse subspace clustering and show that a simple post-processing procedure is capable of delivering consistent clustering under certain "general position" or "restricted eigenvalue" assumptions. We also show that our condition is almost tight with adversarial noise perturbation by constructing a counter-example. These results provide the first exact clustering guarantee of noisy SSC for subspaces of dimension greater then 3.

Motivation & Objective

• To address the graph connectivity problem in noisy sparse subspace clustering, where standard SSC may over-segment due to disconnected components.
• To provide a theoretical guarantee for exact clustering in noisy SSC under a fully deterministic data model with no statistical assumptions on noise.
• To show that a simple post-processing step—merging subspaces based on angular distance—can recover correct clustering with high probability.
• To demonstrate that the derived conditions are nearly tight via a counter-example under adversarial noise perturbation.
• To establish the first provable exact clustering guarantee for noisy SSC when subspace dimension exceeds 3.

Proposed method

• Proposes a post-processing step that computes low-dimensional subspaces for each connected component of the SSC graph and merges those with small angular distance.
• Uses a restricted eigenvalue assumption on the smallest singular value of any d-point subset to ensure robustness to noise.
• Applies a general position assumption to ensure that data points in the same subspace are not linearly dependent in a way that breaks connectivity.
• Employs spectral clustering on the initial sparse graph, followed by a merging phase based on subspace similarity.
• Uses Lasso-based sparse representation to build the initial graph, with a sparsity parameter λ tuned for noise robustness.
• Validates the method on synthetic data with two 4D subspaces in R^n, using normalized data and added Gaussian noise.

Experimental results

Research questions

• RQ1 Can a post-processing step restore graph connectivity in noisy SSC when the original graph is disconnected?
• RQ2 Under what conditions is exact clustering guaranteed for noisy SSC with subspaces of dimension >3?
• RQ3 How tight are the theoretical conditions for exact clustering, especially under adversarial noise?
• RQ4 Can the proposed method achieve perfect clustering without statistical assumptions on the noise?
• RQ5 Is the restricted eigenvalue assumption necessary, or can it be relaxed under stochastic noise models?

Key findings

• The proposed post-processing method achieves 99% clustering accuracy on noiseless data and 93% on noisy data, significantly improving over vanilla Lasso SSC (77% without post-processing).
• The relative violation of the Subspace Detection Property (SEP) is low (0.09 on noisy data), indicating that poor performance is not due to SEP failure but due to graph disconnection.
• A counter-example under adversarial noise shows that the restricted eigenvalue condition is nearly tight, as relaxing it leads to clustering failure.
• The method provides the first exact clustering guarantee for noisy SSC with subspaces of dimension greater than 3 under a fully deterministic model.
• Theoretical analysis shows that under general position or restricted eigenvalue assumptions, the post-processing step recovers the true clustering with high probability.
• The results suggest that the support size of sparse representations can be relaxed below d+1 points, as long as leading subspace directions are preserved for merging.