[Paper Review] Optimality of $\ell_2/\ell_1$-optimization block-length dependent thresholds

Mihailo Stojnic|arXiv (Cornell University)|Mar 29, 2013

Sparse and Compressive Sensing Techniques12 references3 citations

TL;DR

This paper establishes the exact weak threshold for $β$-block-sparse signal recovery using $Â𝔵_2/\u00c2ÂÂÂ_1$-optimization in the linear regime, proving that the lower bounds derived in prior work [39] are tight by deriving matching upper bounds. The result provides a complete, optimal characterization of the performance of block-sparse recovery under i.i.d. Gaussian measurement matrices.

ABSTRACT

The recent work of \cite{CRT,DonohoPol} rigorously proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that $\ell_1$-optimization algorithm succeeds in solving the system. In more recent papers \cite{StojnicCSetamBlock09,StojnicICASSP09block,StojnicJSTSP09} we considered under-determined systems with the so-called extbf{block}-sparse solutions. In a large dimensional and statistical context in \cite{StojnicCSetamBlock09} we determined lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations such that an $\ell_2/\ell_1$-optimization algorithm succeeds in solving the system. These lower bounds happened to be in a solid numerical agreement with what one can observe through numerical experiments. Here we derive the corresponding upper bounds. Moreover, the upper bounds that we obtain in this paper match the lower bounds from \cite{StojnicCSetamBlock09} and ultimately make them optimal.

Motivation & Objective

• To close the gap between previously derived lower bounds and the true performance limits of $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization for block-sparse recovery.
• To derive upper bounds on the weak threshold for block-sparse signal recovery that match the lower bounds from [39], thereby establishing optimality.
• To provide a complete theoretical characterization of the success probability of $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization in the linear regime under i.i.d. Gaussian measurement matrices.
• To unify the lower-bounding framework from [39] with a new upper-bounding mechanism to achieve exact threshold characterization.

Proposed method

• Derives upper bounds on the weak threshold for $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization using a novel theoretical framework based on high-dimensional geometry and concentration of measure.
• Employs a combination of probabilistic analysis and asymptotic analysis in the large-dimensional limit to characterize the boundary between successful and failed recovery.
• Uses the framework from [39] for lower bounds and extends it with new inequalities and approximations to derive matching upper bounds.
• Applies the inverse incomplete gamma function and chi-squared tail probability approximations to model the behavior of block norms in high dimensions.
• Derives a system of equations (50) and (51) that define the weak threshold as the solution to a fixed-point equation involving the block size $d$, sparsity level $β_w$, and measurement ratio $α$.
• Validates the tightness of the bounds by showing that the upper and lower bounds coincide, implying optimality of the threshold.

Experimental results

Research questions

• RQ1 What is the exact weak threshold for $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization in recovering $k$-block-sparse signals under i.i.d. Gaussian measurement matrices?
• RQ2 Can the previously derived lower bounds on the weak threshold be proven tight by establishing matching upper bounds?
• RQ3 How does the block size $d$ influence the optimal recovery threshold in the linear regime?
• RQ4 To what extent does the structure of block-sparsity improve recovery performance compared to standard $ÂÂÂÂ_1$-optimization?

Key findings

• The upper bounds derived in this paper exactly match the lower bounds from [39], proving that the weak threshold for $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization is optimal.
• The exact weak threshold is characterized by the solution to the system of equations (50) and (51), which depend on the block size $d$, the normalized block sparsity $β_w$, and the measurement ratio $α$.
• For any fixed $d$, the optimal threshold is achieved when the system parameters satisfy the equality condition in (50) and (51), ensuring that recovery is possible with overwhelming probability.
• The derived upper bound condition (53) defines the critical point beyond which recovery fails with high probability, confirming the sharpness of the threshold.
• The results confirm that the performance of $ÂÂÂÂ_2/\u00c2ÂÂÂ_1$-optimization in the block-sparse setting is fundamentally limited by the derived threshold, and no improvement is possible under the same assumptions.
• The match between theoretical bounds and numerical simulations, as previously observed in [39], is now rigorously confirmed to be exact, not just approximate.