[Paper Review] Robust Low-Rank Subspace Segmentation with Semidefinite Guarantees

Yuzhao Ni, Ju Sun+3 more|arXiv (Cornell University)|Sep 20, 2010

Sparse and Compressive Sensing Techniques20 references4 citations

TL;DR

This paper proposes LRR-PSD, a robust low-rank subspace segmentation method that enforces symmetric positive semidefinite (PSD) constraints on the affinity matrix during optimization, ensuring better spectral properties and eliminating the need for post-hoc symmetrization. It proves that LRR-PSD is mathematically equivalent to standard LRR in canonical form and offers an efficient, scalable solution via eigen-decomposition, outperforming general SDP solvers in practice.

ABSTRACT

Recently there is a line of research work proposing to employ Spectral Clustering (SC) to segment (group){Throughout the paper, we use segmentation, clustering, and grouping, and their verb forms, interchangeably.} high-dimensional structural data such as those (approximately) lying on subspaces {We follow {liu2010robust} and use the term "subspace" to denote both linear subspaces and affine subspaces. There is a trivial conversion between linear subspaces and affine subspaces as mentioned therein.} or low-dimensional manifolds. By learning the affinity matrix in the form of sparse reconstruction, techniques proposed in this vein often considerably boost the performance in subspace settings where traditional SC can fail. Despite the success, there are fundamental problems that have been left unsolved: the spectrum property of the learned affinity matrix cannot be gauged in advance, and there is often one ugly symmetrization step that post-processes the affinity for SC input. Hence we advocate to enforce the symmetric positive semidefinite constraint explicitly during learning (Low-Rank Representation with Positive SemiDefinite constraint, or LRR-PSD), and show that factually it can be solved in an exquisite scheme efficiently instead of general-purpose SDP solvers that usually scale up poorly. We provide rigorous mathematical derivations to show that, in its canonical form, LRR-PSD is equivalent to the recently proposed Low-Rank Representation (LRR) scheme {liu2010robust}, and hence offer theoretic and practical insights to both LRR-PSD and LRR, inviting future research. As per the computational cost, our proposal is at most comparable to that of LRR, if not less. We validate our theoretic analysis and optimization scheme by experiments on both synthetic and real data sets.

Motivation & Objective

• To address the lack of spectral control and reliance on post-processing symmetrization in existing sparse-reconstruction-based affinity matrix learning methods for subspace segmentation.
• To develop a method that explicitly enforces symmetric positive semidefinite (PSD) constraints during affinity matrix learning to improve theoretical and practical robustness.
• To provide a computationally efficient alternative to general-purpose semidefinite programming (SDP) solvers for low-rank subspace segmentation.
• To establish a theoretical equivalence between the proposed LRR-PSD and the original Low-Rank Representation (LRR) framework in their canonical forms.
• To validate the method’s performance and scalability on both synthetic and real-world datasets, demonstrating improved robustness and efficiency.

Proposed method

• Proposes LRR-PSD, a variant of Low-Rank Representation that explicitly enforces the symmetric positive semidefinite (PSD) constraint on the affinity matrix during optimization.
• Reformulates the LRR problem as a convex optimization problem with a nuclear norm penalty and a PSD constraint, ensuring the solution is inherently symmetric and positive semidefinite.
• Derives an equivalent optimization form that replaces the original SDP formulation with a problem solvable via eigen-decomposition, significantly improving computational efficiency.
• Uses the dual description of the nuclear norm and leverages the fact that the optimal solution to the PSD-constrained problem can be obtained via spectral thresholding on the symmetric part of the data matrix.
• Employs a closed-form solution for the PSD-constrained subproblem, avoiding iterative SDP solvers and enabling faster convergence.
• Introduces a robust version of LRR-PSD that incorporates noise modeling, maintaining the same computational efficiency while improving performance on noisy data.

Experimental results

Research questions

• RQ1 Can enforcing a symmetric positive semidefinite (PSD) constraint during affinity matrix learning improve the spectral properties and robustness of subspace segmentation?
• RQ2 Is the proposed LRR-PSD method computationally more efficient than general-purpose semidefinite programming (SDP) solvers for low-rank subspace segmentation?
• RQ3 Does LRR-PSD maintain theoretical equivalence to the original LRR framework in its canonical form?
• RQ4 Can the robust version of LRR-PSD achieve better performance on noisy or corrupted data compared to standard LRR?
• RQ5 Why do the resulting affinity matrices from LRR-PSD often have only positive eigenvalues and remain close to symmetric in practice?

Key findings

• LRR-PSD is theoretically equivalent to the original LRR in its canonical form, establishing a strong theoretical foundation for both methods.
• The proposed method achieves computational efficiency comparable to or better than standard LRR, with fewer iterations and lower running time due to the use of eigen-decomposition instead of SVD.
• Empirical results show that LRR-PSD consistently produces affinity matrices with only positive eigenvalues and high symmetry, even without explicit symmetrization.
• The robust variant of LRR-PSD performs favorably on noisy data, maintaining high segmentation accuracy with minimal increase in computational cost.
• The method eliminates the need for post-processing symmetrization steps commonly used in spectral clustering pipelines, streamlining the workflow.
• Experiments on synthetic and real datasets confirm that LRR-PSD is more scalable and flexible for large-scale subspace segmentation tasks.