[Paper Review] Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-rank Matrices

Tommaso Cai, Anru R. Zhang|arXiv (Cornell University)|Jun 5, 2013

Sparse and Compressive Sensing Techniques22 references4 citations

TL;DR

This paper establishes sharp restricted isometry conditions for exact and stable recovery of sparse signals and low-rank matrices using constrained ℓ₁ and nuclear norm minimization. It introduces a novel sparse representation of polytopes via convex combinations of sparse vectors, proving that δₜₖᴬ < √((t−1)/t) guarantees exact recovery for all k-sparse signals and matrices with rank ≤ r, and that this bound is tight up to ε for large k.

ABSTRACT

This paper considers compressed sensing and affine rank minimization in both noiseless and noisy cases and establishes sharp restricted isometry conditions for sparse signal and low-rank matrix recovery. The analysis relies on a key technical tool which represents points in a polytope by convex combinations of sparse vectors. The technique is elementary while leads to sharp results. It is shown that for any given constant $t\ge {4/3}$, in compressed sensing $δ_{tk}^A &lt; \sqrt{(t-1)/t}$ guarantees the exact recovery of all $k$ sparse signals in the noiseless case through the constrained $\ell_1$ minimization, and similarly in affine rank minimization $δ_{tr}^\mathcal{M}&lt; \sqrt{(t-1)/t}$ ensures the exact reconstruction of all matrices with rank at most $r$ in the noiseless case via the constrained nuclear norm minimization. Moreover, for any $ε&gt;0$, $δ_{tk}^A

Motivation & Objective

• To establish sharp restricted isometry conditions for exact and stable recovery of sparse signals and low-rank matrices in compressed sensing and affine rank minimization.
• To develop a novel technical tool based on representing points in a polytope as convex combinations of sparse vectors to analyze recovery guarantees.
• To prove that the condition δₜₖᴬ < √((t−1)/t) is both sufficient and necessary (up to ε) for exact recovery of k-sparse signals in the noiseless case.
• To extend the analysis to low-rank matrix recovery, showing δₜᵣᴹ < √((t−1)/t) ensures exact and stable recovery under similar conditions.
• To demonstrate the tightness of the bound by constructing counterexamples where δₜₖᴬ < √((t−1)/t) + ε fails for large k, proving optimality of the threshold.

Proposed method

• Introduces a key technical tool: representing any point in a polytope as a convex combination of sparse vectors, enabling analysis of recovery conditions.
• Uses the restricted isometry constant (RIC) δₜₖᴬ for measurement matrices A and δₜᵣᴹ for linear maps ℳ to quantify stability and recovery performance.
• Applies the sparse representation technique to derive sufficient conditions for exact recovery via constrained ℓ₁ minimization in compressed sensing.
• Extends the framework to affine rank minimization by defining RIC for linear maps ℳ acting on low-rank matrices.
• Employs duality and geometric arguments to show that δₜₖᴬ < √((t−1)/t) implies exact recovery, and constructs counterexamples to prove necessity.
• Leverages results from Cai and Zhang (2010) on restricted orthogonal constants (ROC) to bound the RIC and derive sufficient conditions.

Experimental results

Research questions

• RQ1 What is the tightest possible restricted isometry condition that guarantees exact recovery of all k-sparse signals via ℓ₁ minimization?
• RQ2 Can the same RIC threshold be extended to low-rank matrix recovery using nuclear norm minimization?
• RQ3 Is the condition δₜₖᴬ < √((t−1)/t) necessary for exact recovery, or can it be relaxed?
• RQ4 How does the recovery performance degrade in the presence of noise, and what RIC condition ensures stable recovery?
• RQ5 Can counterexamples be constructed to show that slightly weaker RIC conditions fail for large k?

Key findings

• For any t ≥ 4/3, δₜₖᴬ < √((t−1)/t) is sufficient for exact recovery of all k-sparse signals in the noiseless case via constrained ℓ₁ minimization.
• Similarly, δₜᵣᴹ < √((t−1)/t) ensures exact recovery of all matrices of rank ≤ r in the noiseless case via constrained nuclear norm minimization.
• The bound δₜₖᴬ < √((t−1)/t) is sharp: for any ε > 0, δₜₖᴬ < √((t−1)/t) + ε is not sufficient to guarantee exact recovery for large k.
• The same RIC thresholds are also sufficient for stable recovery of approximately sparse signals and low-rank matrices in the noisy case.
• Counterexamples are constructed where δₜₖᴬ < √((t−1)/t) + ε fails to recover k-sparse signals, proving the tightness of the bound.
• The analysis confirms that the condition δₜₖᴬ < √((t−1)/t) is both necessary and sufficient for exact recovery, establishing optimality in the sense of the best possible threshold.