[Paper Review] Robust Principal Component Analysis?

Emmanuel J. Candès, Xiaodong Li+2 more|ArXiv.org|Dec 18, 2009

Sparse and Compressive Sensing TechniquesEngineering305 citations

TL;DR

This paper proposes Robust Principal Component Analysis (Robust PCA) via Principal Component Pursuit, a convex optimization method that exactly recovers a low-rank matrix and a sparse matrix from their superposition by minimizing a weighted combination of nuclear norm and ℓ₁-norm. The key contribution is a provably correct and scalable solution for robust matrix decomposition under mild conditions, enabling exact recovery even when a constant fraction of entries are arbitrarily corrupted.

ABSTRACT

This paper is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low-rank component and a sparse component. Can we recover each component individually? We prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the L1 norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.

Motivation & Objective

• To address the brittleness of classical PCA in the presence of gross corruptions, such as outliers or missing data.
• To develop a scalable, provably correct method for decomposing a data matrix into low-rank and sparse components when both are unknown in structure and magnitude.
• To enable robust dimensionality reduction and principal component estimation in high-dimensional data with arbitrary corruptions.
• To provide a principled framework for applications such as video surveillance and face recognition where background modeling and outlier removal are critical.

Proposed method

• Formulate the matrix decomposition problem as a convex optimization problem: minimize λ‖L‖⁎ + ‖S‖₁ subject to M = L + S, where L is low-rank and S is sparse.
• Use nuclear norm minimization (‖L‖⁎) to promote low-rank structure and ℓ₁-norm minimization (‖S‖₁) to promote sparsity.
• Leverage duality and random matrix theory to prove exact recovery under mild assumptions on the rank of L and sparsity of S.
• Establish that the solution to the convex program recovers both components exactly when the low-rank and sparse components are sufficiently incoherent and the number of corrupted entries is small relative to the matrix size.
• Design an alternating direction method of multipliers (ADMM) algorithm to solve the optimization problem efficiently in practice.
• Use probabilistic arguments and covering arguments to bound operator norms and ensure recovery guarantees with high probability.

Experimental results

Research questions

• RQ1 Can we exactly recover a low-rank matrix and a sparse matrix from their superposition when the sparse component has arbitrarily large entries and unknown support?
• RQ2 Is there a convex optimization method that can provably recover both components under mild assumptions on the rank and sparsity?
• RQ3 Can the proposed method handle cases where a constant fraction of entries are corrupted, and still achieve exact recovery?
• RQ4 How does the method perform in the presence of missing data, and can it be extended to matrix completion with gross corruptions?
• RQ5 What are the theoretical conditions under which the nuclear norm and ℓ₁-norm minimization jointly recover the true components?

Key findings

• The paper proves that under mild incoherence and sparsity assumptions, the low-rank and sparse components can be exactly recovered with high probability via Principal Component Pursuit.
• Exact recovery is guaranteed when the number of corrupted entries is O(n^{1.5}/log n) for an n×n matrix, assuming the rank is small and the sparse component is sufficiently sparse.
• The nuclear norm minimization promotes low-rank structure, while ℓ₁-norm minimization promotes sparsity, and their combination enables exact decomposition.
• The recovery is robust to gross corruptions: even if a constant fraction of entries are arbitrarily corrupted, the method still recovers the true components.
• The algorithm achieves exact recovery with high probability when the sampling rate of the sparse component is sufficiently low and the low-rank component is incoherent with the standard basis.
• Numerical experiments in video surveillance and face recognition demonstrate that the method successfully separates background (low-rank) from moving objects (sparse) and removes shadows and specularities.