[Paper Review] RES-PCA: A Scalable Approach to Recovering Low-rank Matrices

Chong Peng, Chenglizhao Chen+3 more|arXiv (Cornell University)|Apr 16, 2019

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper proposes RES-PCA, a scalable robust PCA method that recovers low-rank matrices without computing singular value decompositions (SVDs), leveraging group structural information in data to achieve linear computational complexity in both data size and dimension. The method outperforms state-of-the-art approaches like AltProj in speed and scalability, especially when the true rank is unknown, and demonstrates strong performance in low-rank recovery, shadow removal, and anomaly detection across real-world datasets.

ABSTRACT

Robust principal component analysis (RPCA) has drawn significant attentions due to its powerful capability in recovering low-rank matrices as well as successful appplications in various real world problems. The current state-of-the-art algorithms usually need to solve singular value decomposition of large matrices, which generally has at least a quadratic or even cubic complexity. This drawback has limited the application of RPCA in solving real world problems. To combat this drawback, in this paper we propose a new type of RPCA method, RES-PCA, which is linearly efficient and scalable in both data size and dimension. For comparison purpose, AltProj, an existing scalable approach to RPCA requires the precise knowlwdge of the true rank; otherwise, it may fail to recover low-rank matrices. By contrast, our method works with or without knowing the true rank; even when both methods work, our method is faster. Extensive experiments have been performed and testified to the effectiveness of proposed method quantitatively and in visual quality, which suggests that our method is suitable to be employed as a light-weight, scalable component for RPCA in any application pipelines.

Motivation & Objective

• To address the high computational complexity of existing RPCA methods that rely on SVDs, which scale quadratically or cubically with data size.
• To develop a scalable, lightweight RPCA method that remains effective even without prior knowledge of the true matrix rank.
• To exploit underlying group structures in low-rank components to replace SVD-based optimization, enabling linear-time computation.
• To evaluate the method’s performance in real-world applications such as shadow removal and anomaly detection.

Proposed method

• RES-PCA formulates RPCA as a low-rank plus sparse matrix decomposition without relying on SVD computation.
• It uses an augmented Lagrangian method (ALM) for optimization, avoiding matrix decompositions at each iteration.
• The method leverages geometric and group structural similarities among data points to implicitly model low-rank structure.
• It replaces the nuclear norm with a group-based regularization that captures collective low-rank patterns.
• The algorithm achieves linear complexity in both sample size $n$ and dimension $d$, making it scalable for large-scale data.
• It does not require the true rank as input, unlike methods such as AltProj, which fail when rank is unknown.

Experimental results

Research questions

• RQ1 Can a robust PCA method be designed to avoid SVD computation while maintaining high recovery accuracy?
• RQ2 How does the performance of RES-PCA compare to state-of-the-art methods like AltProj when the true rank is unknown?
• RQ3 To what extent does exploiting group structure in data improve low-rank recovery efficiency and scalability?
• RQ4 Can RES-PCA effectively handle real-world tasks such as shadow removal and anomaly detection with linear-time complexity?

Key findings

• RES-PCA achieves linear time complexity in both data dimension $d$ and sample size $n$, confirmed empirically through runtime measurements on multiple datasets.
• The method outperforms AltProj in speed even when AltProj has access to the true rank, and succeeds where AltProj fails when the rank is unknown.
• In face image shadow removal, RES-PCA successfully removes shadows from both subject 1 and subject 2, while other methods fail on subject 1.
• For anomaly detection on USPS digits, RES-PCA correctly identifies all '7's and several deviant '1's as outliers, as shown by high $\ell_2$ norms in the sparse component $S$.
• The average number of iterations across all experiments was 23–25, with convergence achieved within 30 iterations, indicating stable and efficient optimization.
• Visual and quantitative results confirm that RES-PCA delivers superior recovery quality and scalability compared to existing SVD-based RPCA methods.