[Paper Review] Accelerated Sparse Subspace Clustering

Abolfazl Hashemi, Haris Vikalo|arXiv (Cornell University)|Oct 31, 2017

Face and Expression Recognition18 references5 citations

TL;DR

This paper proposes Accelerated Sparse Subspace Clustering (ASSC), a fast and accurate algorithm that uses an accelerated orthogonal least-squares variant to construct a subspace-preserving similarity matrix for clustering high-dimensional data lying on a union of subspaces. ASSC achieves runtime comparable to OMP-based methods while significantly outperforming them in accuracy, especially under subspace overlap, and matches BP-based methods in accuracy with orders-of-magnitude speedup.

ABSTRACT

State-of-the-art algorithms for sparse subspace clustering perform spectral clustering on a similarity matrix typically obtained by representing each data point as a sparse combination of other points using either basis pursuit (BP) or orthogonal matching pursuit (OMP). BP-based methods are often prohibitive in practice while the performance of OMP-based schemes are unsatisfactory, especially in settings where data points are highly similar. In this paper, we propose a novel algorithm that exploits an accelerated variant of orthogonal least-squares to efficiently find the underlying subspaces. We show that under certain conditions the proposed algorithm returns a subspace-preserving solution. Simulation results illustrate that the proposed method compares favorably with BP-based method in terms of running time while being significantly more accurate than OMP-based schemes.

Motivation & Objective

• To address the computational inefficiency of basis pursuit (BP)-based sparse subspace clustering, which is slow for large-scale data.
• To overcome the poor performance of OMP-based methods in scenarios with highly similar subspaces or subspace overlap.
• To develop a scalable, fast, and accurate alternative to existing SSC algorithms that maintains subspace-preserving properties.
• To provide theoretical justification for subspace preservation under independent subspace assumptions.

Proposed method

• The method employs an accelerated variant of orthogonal least-squares (OLS) to compute sparse representations of each data point using other points as a dictionary.
• It constructs a similarity matrix by solving a sparse representation problem with a greedy, iterative selection of atoms from the data matrix.
• The algorithm uses a modified residual update and rank-revealing QR to enhance numerical stability and convergence speed.
• The resulting representation matrix is used as a similarity matrix for spectral clustering to obtain the final clustering solution.
• The method is designed to be scalable and efficient, with computational complexity significantly lower than BP-based approaches.
• Theoretical analysis shows that under independent subspace conditions, the solution is subspace-preserving.

Experimental results

Research questions

• RQ1 Can a fast, greedy algorithm like accelerated OLS achieve subspace-preserving representation comparable to convex optimization (BP) in sparse subspace clustering?
• RQ2 How does the performance of the proposed method compare to BP-based and OMP-based SSC in terms of clustering accuracy and runtime?
• RQ3 Does the proposed method maintain robustness when subspaces are not independent and data points from different subspaces are highly similar?
• RQ4 Under what conditions does the accelerated OLS method produce a subspace-preserving solution?
• RQ5 Can the proposed method scale efficiently to large-scale datasets while preserving clustering accuracy?

Key findings

• ASSC achieves runtime comparable to SSC-OMP and is orders of magnitude faster than SSC-BP, especially on large datasets with up to 5,000 points.
• In scenarios with independent subspaces, ASSC achieves a subspace preserving rate and error comparable to SSC-BP, with significantly better accuracy than SSC-OMP.
• When subspaces are dependent or data points are highly similar, SSC-OMP performance degrades severely, but ASSC maintains high accuracy and robustness.
• ASSC with L=2 achieves superior clustering accuracy compared to both SSC-BP and SSC-OMP across all tested settings.
• The subspace preserving error for ASSC is consistently lower than for SSC-OMP and comparable to SSC-BP, confirming its theoretical guarantees under independent subspaces.
• The method is scalable and suitable for large-scale data, as demonstrated by consistent performance across data sizes from 250 to 5,000 points.