[Paper Review] Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

HanQin Cai, Jialin Liu+1 more|arXiv (Cornell University)|Oct 11, 2021

Sparse and Compressive Sensing Techniques48 references20 citations

TL;DR

This paper proposes Learned Robust PCA (LRPCA), a scalable deep unfolding method for high-dimensional outlier detection that replaces iterative singular value thresholding with a learnable feedforward-recurrent-mixed neural network, enabling infinite-iteration unfolding and end-to-end parameter learning. LRPCA achieves superior performance over state-of-the-art methods like ScaledGD and AltProj in both synthetic and real-world video background subtraction tasks, with strong generalization and recovery guarantees under mild assumptions.

ABSTRACT

Robust principal component analysis (RPCA) is a critical tool in modern machine learning, which detects outliers in the task of low-rank matrix reconstruction. In this paper, we propose a scalable and learnable non-convex approach for high-dimensional RPCA problems, which we call Learned Robust PCA (LRPCA). LRPCA is highly efficient, and its free parameters can be effectively learned to optimize via deep unfolding. Moreover, we extend deep unfolding from finite iterations to infinite iterations via a novel feedforward-recurrent-mixed neural network model. We establish the recovery guarantee of LRPCA under mild assumptions for RPCA. Numerical experiments show that LRPCA outperforms the state-of-the-art RPCA algorithms, such as ScaledGD and AltProj, on both synthetic datasets and real-world applications.

Motivation & Objective

• To address the high computational cost of existing RPCA methods that rely on expensive singular value thresholding (SVT) at every iteration.
• To overcome the limitation of finite-iteration deep unfolding, which requires retraining for higher accuracy.
• To develop a scalable, learnable, and theoretically grounded approach for high-dimensional RPCA problems.
• To enable generalization across different matrix sizes and ranks without retraining.
• To establish recovery guarantees for the proposed method under mild assumptions.

Proposed method

• Proposes a feedforward-recurrent-mixed neural network (FRMNN) to model infinite-iteration unfolding, allowing arbitrary accuracy without retraining.
• Replaces the singular value thresholding (SVT) step in classical RPCA with learnable, adaptive thresholding and step size parameters.
• Employs deep unfolding to parameterize a non-convex RPCA algorithm, enabling end-to-end backpropagation-based learning of free parameters.
• Introduces a layer-wise training strategy with stochastic gradient descent (SGD), using synthetic data with known ground truth for supervision.
• Applies scaling of thresholding parameters during testing to generalize across different matrix sizes and ranks, based on theoretical bounds.
• Uses a hybrid training scheme: layer-wise training with SGD and grid search for hyperparameters, with validation on held-out synthetic instances.

Experimental results

Research questions

• RQ1 Can a deep unfolding approach for RPCA avoid the computational bottleneck of singular value thresholding (SVT) in high-dimensional settings?
• RQ2 Can a learnable, infinite-iteration unfolding framework be designed to support arbitrary accuracy without retraining?
• RQ3 Does the proposed method generalize well across different matrix dimensions and ranks when trained on a single base setting?
• RQ4 Can the learned parameters be theoretically justified with recovery guarantees under mild assumptions?
• RQ5 How does the performance of the proposed method compare to state-of-the-art RPCA algorithms like ScaledGD and AltProj?

Key findings

• LRPCA outperforms ScaledGD and AltProj on both synthetic datasets and real-world video background subtraction tasks, achieving higher accuracy with fewer iterations.
• The model generalizes well: when trained on a base setting (n=1000, r=5), it achieves 7 iterations to reach 10−4 accuracy on n=3000, r=5, compared to 6 iterations when trained on the target setting.
• Trained step sizes are large in early iterations and decay gradually, aligning with theoretical expectations and enabling aggressive initial convergence.
• Thresholding parameters decay exponentially, consistent with the theoretical bound in Lemma 10, indicating stable and adaptive outlier suppression.
• Training time scales sublinearly with problem size due to GPU parallelism, with 2405 seconds for n=5000, r=5, α=0.1.
• The method achieves exact recovery under mild assumptions, including bounded incoherence and sparsity, with theoretical recovery guarantees established.