[Paper Review] Compressive Principal Component Pursuit

John Wright, Arvind Ganesh+2 more|arXiv (Cornell University)|Feb 21, 2012

Sparse and Compressive Sensing Techniques26 references20 citations

TL;DR

This paper proposes a compressive principal component pursuit method to recover low-rank and sparse matrix components from a small number of random linear measurements. It proves that exact recovery is achievable when the number of measurements exceeds the intrinsic degrees of freedom by a polylogarithmic factor, using a convex optimization framework that minimizes nuclear and $$\ell^1$ norms under measurement constraints.

ABSTRACT

We consider the problem of recovering a target matrix that is a superposition of low-rank and sparse components, from a small set of linear measurements. This problem arises in compressed sensing of structured high-dimensional signals such as videos and hyperspectral images, as well as in the analysis of transformation invariant low-rank recovery. We analyze the performance of the natural convex heuristic for solving this problem, under the assumption that measurements are chosen uniformly at random. We prove that this heuristic exactly recovers low-rank and sparse terms, provided the number of observations exceeds the number of intrinsic degrees of freedom of the component signals by a polylogarithmic factor. Our analysis introduces several ideas that may be of independent interest for the more general problem of compressed sensing and decomposing superpositions of multiple structured signals.

Motivation & Objective

• To address the problem of recovering low-rank and sparse matrix components from highly compressive linear measurements.
• To extend robust principal component analysis (RPCA) to the compressive sensing regime where only partial measurements are available.
• To establish theoretical conditions under which the convex relaxation of the low-rank and sparse decomposition problem achieves exact recovery.
• To analyze the performance of the natural convex heuristic for compressive robust matrix recovery under random measurement ensembles.
• To provide a theoretical foundation for compressed sensing of structured signals such as videos and hyperspectral images.

Proposed method

• Formulates the compressive robust matrix recovery problem as a convex optimization problem minimizing the nuclear norm of the low-rank component and the $$\ell^1$ norm of the sparse component under linear measurement constraints.
• Uses a random measurement subspace $Q$ that is incoherent with both the low-rank and sparse components with high probability.
• Applies duality-based analysis to construct a dual certificate composed of three parts: $\mathbf{W}^L$, $\mathbf{W}^S$, and $\mathbf{W}^T$, ensuring optimality of the solution.
• Employs Neumann series to construct the dual variable $\mathbf{W}^S$ satisfying constraints on the sparse support and orthogonality to the tangent space of the low-rank component.
• Uses Bernstein’s inequality and concentration bounds to control the Frobenius norm of the dual certificate components, particularly $\mathbf{W}^L$, under random sampling.
• Establishes recovery guarantees by showing that the dual certificate satisfies the required conditions with high probability when the number of measurements exceeds the intrinsic degrees of freedom by a polylogarithmic factor.

Experimental results

Research questions

• RQ1 Can low-rank and sparse matrix components be exactly recovered from a small number of random linear measurements using convex optimization?
• RQ2 What is the minimal number of measurements required for exact recovery of the low-rank and sparse components in the compressive setting?
• RQ3 How does the performance of the convex heuristic for compressive principal component pursuit scale with the rank and sparsity of the components?
• RQ4 Under what conditions does the dual certificate construction succeed in certifying the optimality of the solution?
• RQ5 To what extent can the number of measurements be reduced below the full matrix dimension while still ensuring exact recovery?

Key findings

• Exact recovery of both low-rank and sparse components is guaranteed when the number of measurements exceeds the intrinsic degrees of freedom by a polylogarithmic factor.
• The recovery condition holds under random measurement ensembles that are incoherent with both the low-rank and sparse components with high probability.
• The dual certificate construction succeeds with high probability when the rank $r$ satisfies $r \leq c_r n / (\mu \log^2 m)$, where $c_r$ is a sufficiently small constant.
• The Frobenius norm of the dual certificate $\mathbf{W}^L$ is bounded by $3\sqrt{r}$, which ensures the optimality of the solution under the derived conditions.
• The method achieves exact recovery even when the number of measurements is as low as half the number of matrix entries, provided the rank and sparsity are sufficiently small.
• The theoretical analysis introduces new techniques in random matrix theory and duality that may be applicable to broader problems in compressed sensing of structured signals.