[Paper Review] Rank-Sparsity Incoherence for Matrix Decomposition

Venkat Chandrasekaran, Sujay Sanghavi+2 more|Jun 11, 2009

Sparse and Compressive Sensing TechniquesEngineering27 references91 citations

TL;DR

This paper proposes a convex optimization approach to decompose a matrix into sparse and low-rank components by minimizing a combination of the ℓ₁ norm and nuclear norm. It introduces the concept of rank-sparsity incoherence as an uncertainty principle between sparsity patterns and row/column spaces, establishing deterministic sufficient conditions for exact recovery and showing high probability recovery for random matrix ensembles.

ABSTRACT

Suppose we are given a matrix that is formed by adding an unknown sparse matrix to an unknown low-rank matrix. Our goal is to decompose the given matrix into its sparse and low-rank components. Such a problem arises in a number of applications in model and system identification, and is NP-hard in general. In this paper we consider a convex optimization formulation to splitting the specified matrix into its components, by minimizing a linear combination of the $\ell_1$ norm and the nuclear norm of the components. We develop a notion of \emph{rank-sparsity incoherence}, expressed as an uncertainty principle between the sparsity pattern of a matrix and its row and column spaces, and use it to characterize both fundamental identifiability as well as (deterministic) sufficient conditions for exact recovery. Our analysis is geometric in nature, with the tangent spaces to the algebraic varieties of sparse and low-rank matrices playing a prominent role. When the sparse and low-rank matrices are drawn from certain natural random ensembles, we show that the sufficient conditions for exact recovery are satisfied with high probability. We conclude with simulation results on synthetic matrix decomposition problems.

Motivation & Objective

• To address the fundamental challenge of decomposing a matrix into its sparse and low-rank components when no prior information is available about the sparsity pattern or rank.
• To overcome the ill-posed nature of the decomposition problem by introducing a geometric condition—rank-sparsity incoherence—that ensures identifiability.
• To provide deterministic sufficient conditions for exact recovery of sparse and low-rank components using convex relaxation.
• To show that under random matrix ensembles, the proposed conditions are satisfied with high probability, enabling reliable recovery.

Proposed method

• Formulates the matrix decomposition problem as a convex optimization problem minimizing a weighted sum of the ℓ₁ norm of the sparse component and the nuclear norm of the low-rank component.
• Introduces the rank-sparsity incoherence condition as an uncertainty principle between the sparsity pattern of a matrix and its row/column spaces.
• Defines the tangent space to the algebraic variety of low-rank matrices and uses it to characterize the geometry of the recovery problem.
• Uses the quantity ξ(M), the maximum ∞-norm of unit-normed elements in the tangent space of a low-rank matrix M, to quantify how diffuse or concentrated the matrix’s structure is.
• Introduces the degree measure deg_max(A) and deg_min(A) for a sparse matrix A to control the concentration of its non-zero entries across rows and columns.
• Employs semidefinite programming to solve the convex relaxation and derives conditions under which the original sparse and low-rank components are uniquely recovered.

Experimental results

Research questions

• RQ1 Under what conditions can a matrix be uniquely decomposed into a sparse and a low-rank component?
• RQ2 How can we characterize the geometric relationship between the sparsity pattern of a matrix and its row/column spaces to ensure identifiability?
• RQ3 What deterministic conditions guarantee exact recovery of the sparse and low-rank components via convex relaxation?
• RQ4 How likely is exact recovery when the sparse and low-rank components are drawn from natural random ensembles?
• RQ5 What role does the tangent space to the algebraic variety of low-rank matrices play in determining recovery feasibility?

Key findings

• The paper establishes that if the low-rank matrix B satisfies ξ(B) ≤ 2 inc(B), where inc(B) measures the alignment of its row and column spaces with the standard basis, then the tangent space elements are not too sparse, preventing identifiability issues.
• The sufficient condition for exact recovery involves the rank-sparsity incoherence measure, which ensures that the sparse and low-rank components are not aligned in a way that makes them indistinguishable.
• For random matrices drawn from natural ensembles, the sufficient conditions for exact recovery are satisfied with high probability, particularly when the rank and sparsity levels are not too large relative to the matrix dimensions.
• The degree measure deg_max(A) of a sparse matrix A is bounded below by 2λ, where λ is the largest singular value of the matrix formed by the non-zero entries of A, linking sparsity concentration to recovery feasibility.
• The quantity μ(A), defined via an optimization over unitary matrices, satisfies μ(A) ≤ deg_max(A) and μ(A) ≥ deg_min(A), providing bounds on the sparsity concentration of A.
• The paper proves that ξ(B) ≤ 2 inc(B) and ξ(B) ≥ max(β(U), β(V)), where β(U) and β(V) are measures of alignment between the row and column spaces of B and the standard basis vectors, providing tight bounds on the incoherence measure.