[Paper Review] Achieving stable subspace clustering by post-processing generic clustering results

Duc-Son Pham, Ognjen Arandjelović+1 more|arXiv (Cornell University)|May 27, 2016

Face and Expression Recognition43 references4 citations

TL;DR

This paper proposes a post-processing method to stabilize and improve sparse subspace clustering (SSC) results by identifying and reassigning misclustered points using stable, randomly sampled subspaces. By computing residual errors and applying a conservative dominant nearest subspace classification, the method reduces clustering errors significantly with minimal disruption to correctly assigned points.

ABSTRACT

We propose an effective subspace selection scheme as a post-processing step to improve results obtained by sparse subspace clustering (SSC). Our method starts by the computation of stable subspaces using a novel random sampling scheme. Thus constructed preliminary subspaces are used to identify the initially incorrectly clustered data points and then to reassign them to more suitable clusters based on their goodness-of-fit to the preliminary model. To improve the robustness of the algorithm, we use a dominant nearest subspace classification scheme that controls the level of sensitivity against reassignment. We demonstrate that our algorithm is convergent and superior to the direct application of a generic alternative such as principal component analysis. On several popular datasets for motion segmentation and face clustering pervasively used in the sparse subspace clustering literature the proposed method is shown to reduce greatly the incidence of clustering errors while introducing negligible disturbance to the data points already correctly clustered.

Motivation & Objective

• Improve the robustness and accuracy of sparse subspace clustering (SSC) on real-world datasets like motion segmentation and face clustering.
• Address the challenge of parameter sensitivity and suboptimal clustering results in SSC due to default or non-optimized hyperparameters.
• Develop a generic, non-application-specific post-processing technique that enhances clustering without altering the original algorithm’s core.
• Minimize reassignment errors while correcting misclustered points through a stable subspace estimation and residual-based evaluation framework.

Proposed method

• Construct stable subspaces for each initial cluster via random sampling of data points, ensuring robustness against noise and outliers.
• Compute residual errors between each data point and the preliminary subspaces using ℓp norms to assess goodness-of-fit.
• Apply a dominant nearest subspace classification strategy to decide reassignment, using a conservative threshold to limit false corrections.
• Use regularization and stability selection principles to enhance subspace estimation and reduce sensitivity to initialization.
• Integrate the method as a post-processing step after any SSC variant, requiring no changes to the original clustering pipeline.
• Control reassignment sensitivity via a hyperparameter η, with conservative settings prioritizing reliability over aggressive correction.

Experimental results

Research questions

• RQ1 Can a post-processing method improve the clustering accuracy of sparse subspace clustering without modifying its core algorithm?
• RQ2 How effective is stable subspace estimation via random sampling in identifying and correcting misclustered data points?
• RQ3 To what extent does the proposed method reduce clustering errors compared to baseline SSC and oracle-based subspace knowledge?
• RQ4 How does the conservative dominant nearest subspace classification strategy prevent over-correction and maintain stability?
• RQ5 Can the method generalize across diverse datasets such as motion segmentation and face clustering with varying numbers of classes?

Key findings

• On the Hopkins 155 motion segmentation dataset, the proposed method significantly reduces clustering errors, approaching the performance bound of oracle subspace knowledge.
• On the Yale B dataset, the method reduces error from 6.25% (SSC) to 4.69% (SSS) for 2 classes, and from 8.33% to 5.63% for 3 classes.
• On the PIE dataset, error drops from 2.50% (SSC) to 2.50% (SSS) for 2 classes, but remains unchanged due to already high initial accuracy, indicating conservative behavior.
• For 8-class clustering on Yale B, the method corrects 4 points (out of 128), reducing error from 5.66% to 4.88%, demonstrating effectiveness even at high class counts.
• The number of reassignments made by the stable subspace selection (SSS) is consistently close to that of the oracle subspace method (OSS), indicating high efficiency.
• The method maintains high normalized mutual information (NMI), with SSS achieving 0.9059 NMI on Yale B (8 classes), compared to 0.8866 for SSC, showing improved clustering quality.