[Paper Review] Deterministic Bounds for Restricted Isometry of Compressed Sensing Matrices

Shriram Sarvotham, Richard G. Baraniuk|arXiv (Cornell University)|Mar 16, 2011

Sparse and Compressive Sensing Techniques26 references3 citations

TL;DR

This paper establishes deterministic upper and lower bounds on the Restricted Isometry Property (RIP) constant for compressed sensing matrices, using structural analysis of submatrix singular values and geometric packing/covering arguments on Grassmannian spaces. It reveals a significant performance gap between random Gaussian matrices and theoretically optimal matrices, demonstrating that better RIP ratios are achievable through structured designs.

ABSTRACT

Compressed Sensing (CS) is an emerging field that enables reconstruction of a sparse signal $x \in {\mathbb R} ^n$ that has only $k \ll n$ non-zero coefficients from a small number $m \ll n$ of linear projections. The projections are obtained by multiplying $x$ by a matrix $Φ\in {\mathbb R}^{m imes n}$ --- called a CS matrix --- where $k < m \ll n$. In this work, we ask the following question: given the triplet $\{k, m, n \}$ that defines the CS problem size, what are the deterministic limits on the performance of the best CS matrix in ${\mathbb R}^{m imes n}$? We select Restricted Isometry as the performance metric. We derive two deterministic converse bounds and one deterministic achievable bound on the Restricted Isometry for matrices in ${\mathbb R}^{m imes n}$ in terms of $n$, $m$ and $k$. The first converse bound (structural bound) is derived by exploiting the intricate relationships between the singular values of sub-matrices and the complete matrix. The second converse bound (packing bound) and the achievable bound (covering bound) are derived by recognizing the equivalence of CS matrices to codes on Grassmannian spaces. Simulations reveal that random Gaussian $Φ$ provide far from optimal performance. The derivation of the three bounds offers several new geometric insights that relate optimal CS matrices to equi-angular tight frames, the Welch bound, codes on Grassmannian spaces, and the Generalized Pythagorean Theorem (GPT).

Motivation & Objective

• To determine the fundamental deterministic limits on the Restricted Isometry Property (RIP) for any m×n compressed sensing matrix given k-sparse signals.
• To identify the theoretical performance ceiling for RIP constants δk across all possible matrices in R^{m×n} for given n, m, and k.
• To derive bounds that reveal the gap between random Gaussian matrices and optimal deterministic constructions.
• To establish geometric connections between optimal CS matrices, equiangular tight frames, Grassmannian codes, and the Generalized Pythagorean Theorem.

Proposed method

• Derives a structural converse bound by analyzing the relationships between the singular values of submatrices and the full matrix Φ.
• Develops a packing converse bound by modeling CS matrices as codes on Grassmannian spaces, leveraging sphere packing in high-dimensional subspaces.
• Establishes an achievable covering bound using covering arguments on Grassmannian spaces, representing the best possible RIP performance.
• Uses polynomial root analysis of a derived characteristic equation to estimate the distribution of singular values in random m×k submatrices of Φ.
• Compares theoretical bounds with empirical performance of Gaussian matrices and known optimal matrices from Conway, Hardin, and Sloane’s Grassmannian packing data.
• Extends the Welch bound to higher-order k > 2 using the structural bound, showing equivalence for k=2.

Experimental results

Research questions

• RQ1 What are the tightest possible deterministic bounds on the Restricted Isometry constant δk for any m×n compressed sensing matrix?
• RQ2 How does the performance of random Gaussian matrices compare to the theoretical limits of optimal CS matrices in terms of RIP?
• RQ3 What geometric and algebraic structures—such as equiangular tight frames or Grassmannian codes—underlie optimal CS matrices?
• RQ4 Can the singular value distribution of random m×k submatrices of Φ be predicted using the parameters of the structural bound?
• RQ5 What is the relationship between the structural bound and the Welch bound for k > 2?

Key findings

• A large performance gap exists between random Gaussian matrices and the theoretical deterministic bounds, indicating that superior CS matrices exist.
• For small n, the structural bound is tighter; for large n, the packing bound becomes the tighter converse bound.
• The structural bound for k=2 is mathematically equivalent to the Welch bound, and the method extends this bound to k > 2.
• The roots of a derived polynomial equation provide accurate estimates of the squared singular values of randomly selected m×k submatrices of Φ.
• Simulations confirm that r_i² values (from the structural bound) closely match the empirical distribution of singular values in submatrices.
• The covering bound provides a deterministic achievable limit on RIP performance, suggesting that optimal matrices can achieve significantly better δk than Gaussian matrices.