[Paper Review] Various thresholds for $\ell_1$-optimization in compressed sensing

Mihailo Stojnic|ArXiv.org|Jul 21, 2009

Sparse and Compressive Sensing Techniques79 references77 citations

TL;DR

This paper provides a refined theoretical analysis of $β$-thresholds for $Ø_1$-optimization in compressed sensing, leveraging random matrix theory and Gaussian process inequalities to derive lower bounds on the strong, weak, and sectional thresholds. It achieves performance bounds that match or improve upon the best-known results from prior work, particularly in the linear regime where sparsity $k = \beta n$ and measurements $m = \alpha n$ scale linearly with $n$. The analysis assumes a random measurement matrix with i.i.d. Gaussian entries and establishes conditions under which $Ø_1$-minimization exactly recovers $k$-sparse signals with overwhelming probability.

ABSTRACT

Recently, \cite{CRT,DonohoPol} theoretically analyzed the success of a polynomial $\ell_1$-optimization algorithm in solving an under-determined system of linear equations. In a large dimensional and statistical context \cite{CRT,DonohoPol} proved that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that $\ell_1$-optimization succeeds in solving the system. In this paper, we provide an alternative performance analysis of $\ell_1$-optimization and obtain the proportionality constants that in certain cases match or improve on the best currently known ones from \cite{DonohoPol,DT}.

Motivation & Objective

• To provide a new theoretical performance analysis of $Ø_1$-optimization in compressed sensing under the linear regime.
• To derive lower bounds on strong, weak, and sectional thresholds for exact recovery of $k$-sparse signals.
• To improve upon or match the best-known threshold constants from prior work, particularly those in [28, 29].
• To analyze the performance of $Ø_1$-optimization under the assumption that the measurement matrix $A$ has a null-space uniformly distributed in the Grassmannian.
• To lay the groundwork for future extensions to approximately sparse signals, noisy measurements, and $\ell_q$-optimization with $0 < q < 1$.

Proposed method

• The analysis assumes a random measurement matrix $A$ with i.i.d. standard Gaussian entries and uses the null-space assumption to model the Grassmannian distribution of the matrix kernel.
• It applies advanced tools from high-dimensional probability, particularly the results of [47], which in turn rely on [68, 20] for tail estimates of Lipschitz functions on the sphere.
• The method derives threshold conditions by analyzing the geometry of the null-space and the feasibility of the $Ø_1$-minimization problem via saddle-point approximations and error function identities.
• Key equations involve solving transcendental equations involving the inverse error function, such as $ (1-ε)(1-\beta_{w}^{+}) \frac{\sqrt{1/(2\pi)} e^{-(\text{erfinv}(2\frac{1-\theta_{w}^{+}}{1-\beta_{w}^{+}}-1))^{2}}}{\theta_{w}^{+}} - \sqrt{2} \text{erfinv}((2\frac{(1+\epsilon)(1-\theta_{w}^{+})}{1-\beta_{w}^{+}}-1)) = 0 $, which define the weak threshold $\theta_{w}^{+}$.
• Theoretical thresholds are derived by ensuring that the solution of the $Ø_1$-problem coincides with the true sparse solution with overwhelming probability, based on concentration of measure and isoperimetric inequalities.
• The framework is general and can be extended to noisy settings, approximately sparse signals, and $\ell_q$-minimization with $0 < q < 1$.

Experimental results

Research questions

• RQ1 What are the precise lower bounds on the strong, weak, and sectional thresholds for $Ø_1$-optimization in the linear regime of compressed sensing?
• RQ2 How do the derived thresholds compare to the best-known results in the literature, particularly those from [28, 29]?
• RQ3 Can the analysis framework be extended to handle noisy measurements or approximately sparse signals?
• RQ4 What is the role of the null-space distribution of the measurement matrix in determining recovery thresholds?
• RQ5 Can the results be adapted to $\ell_q$-optimization with $0 < q < 1$?

Key findings

• The paper derives a new lower bound on the weak threshold $\theta_w^+$ through a system of equations involving the inverse error function, which determines the critical $\alpha$ and $\beta$ values for successful recovery.
• For the case of signed vectors, the derived threshold results match those in [29, 30] across most of the parameter range, with only a narrow region near $\alpha \to 1$ showing improvement.
• The strong threshold results for signed vectors were computed but not included due to complexity and because they underperform the state-of-the-art in most cases.
• The analysis establishes that if $\alpha$ and $\beta_w^+$ satisfy a specific inequality involving error functions and exponential terms, then the solutions of the original system and the $Ø_1$-problem coincide with overwhelming probability.
• The framework is general and can be extended to analyze noisy compressed sensing, approximately sparse signals, and $\ell_q$-optimization with $0 < q < 1$.
• The results are of independent mathematical interest, as they can be applied to determine neighborliness thresholds of projected cross-polytopes, regular simplices, and positive orthants.