[Paper Review] Restricted isometry property of matrices with independent columns and neighborly polytopes by random sampling

Radosław Adamczak, Alexander E. Litvak+2 more|arXiv (Cornell University)|Apr 30, 2009

Sparse and Compressive Sensing Techniques19 references18 citations

TL;DR

This paper establishes that random matrices with independent, isotropic, sub-exponential entries satisfy the restricted isometry property (RIP) with overwhelming probability, enabling exact recovery of $ m $-sparse vectors via $ \ell_1 $-minimization. The key result shows that such matrices achieve RIP of order $ m \leq Cn / \log^2(cN/n) $, implying that the symmetric convex hull of the matrix columns forms an $ m $-centrally neighborly polytope with high probability.

ABSTRACT

This paper considers compressed sensing matrices and neighborliness of a centrally symmetric convex polytope generated by vectors $\pm X_1,...,\pm X_N\in\R^n$, ($N\ge n$). We introduce a class of random sampling matrices and show that they satisfy a restricted isometry property (RIP) with overwhelming probability. In particular, we prove that matrices with i.i.d. centered and variance 1 entries that satisfy uniformly a sub-exponential tail inequality possess this property RIP with overwhelming probability. We show that such "sensing" matrices are valid for the exact reconstruction process of $m$-sparse vectors via $\ell_1$ minimization with $m\le Cn/\log^2 (cN/n)$. The class of sampling matrices we study includes the case of matrices with columns that are independent isotropic vectors with log-concave densities. We deduce that if $K\subset \R^n$ is a convex body and $X_1,..., X_N\in K$ are i.i.d. random vectors uniformly distributed on $K$, then, with overwhelming probability, the symmetric convex hull of these points is an $m$-centrally-neighborly polytope with $m\sim n/\log^2 (cN/n)$.

Motivation & Objective

• To establish sufficient conditions under which random matrices with independent columns satisfy the restricted isometry property (RIP).
• To link the RIP of such matrices to the neighborliness of centrally symmetric convex polytopes formed by their columns.
• To show that $ \ell_1 $-minimization can exactly reconstruct $ m $-sparse vectors when the sensing matrix satisfies RIP with parameter $ \delta_{2m} < \sqrt{2}-1 $.
• To derive sharp bounds on the maximum sparsity $ m $ for which exact recovery is possible with high probability.

Proposed method

• Introduces a class of random matrices with independent, isotropic columns satisfying sub-exponential tail decay and concentration of the $ \ell_2 $-norm.
• Uses the RIP condition $ \delta_{2m}(A/\sqrt{n}) < \sqrt{2}-1 $ as a sufficient condition for exact $ \ell_1 $-reconstruction of $ m $-sparse vectors.
• Applies concentration inequalities and tail estimates for linear forms to bound the isometry constant $ \delta_{2m} $.
• Employs Sudakov-type minoration principles for exponential random variables to derive lower bounds on the expected operator norm of the matrix over sparse vectors.
• Analyzes the geometry of the symmetric convex hull $ K(A) $ of $ \pm X_1, \dots, \pm X_N $, showing it is $ m $-centrally neighborly under the derived conditions.
• Uses probabilistic methods to bound the failure probability of the RIP and neighborliness, showing it decays exponentially in $ n $.

Experimental results

Research questions

• RQ1 Under what conditions on the distribution of independent columns does a random matrix satisfy the restricted isometry property (RIP) with high probability?
• RQ2 What is the maximal sparsity level $ m $ for which $ \ell_1 $-minimization exactly recovers $ m $-sparse vectors using such random matrices?
• RQ3 How does the neighborliness of the symmetric convex hull $ K(A) $ of $ \pm X_1, \dots, \pm X_N $ relate to the RIP of the matrix $ A $?
• RQ4 Can the bound on $ m $ for exact recovery be improved beyond $ m \sim n / \log^2(N/n) $ under sub-exponential tail assumptions?
• RQ5 What is the sharp order of magnitude of the failure probability for the RIP and neighborliness in high dimensions?

Key findings

• Random matrices with i.i.d. centered entries satisfying a uniform sub-exponential tail inequality possess the restricted isometry property of order $ m \leq Cn / \log^2(cN/n) $ with overwhelming probability.
• The symmetric convex hull of $ \pm X_1, \dots, \pm X_N $, where $ X_i $ are i.i.d. isotropic vectors with log-concave densities, is $ m $-centrally neighborly with $ m \sim n / \log^2(cN/n) $ with high probability.
• For matrices with i.i.d. entries satisfying sub-exponential tail decay, exact recovery of $ m $-sparse vectors via $ \ell_1 $-minimization is possible with high probability when $ m \leq Cn / \log^2(cN/n) $.
• The bound on $ m $ is sharp up to numerical constants, as shown by matching lower bounds on the operator norm of the matrix over sparse vectors.
• The failure probability for RIP and neighborliness decays as $ \exp(-c\sqrt{n}) $, indicating overwhelming probability of success as $ n \to \infty $.
• The result applies to uniformly distributed vectors on a convex body $ K \subset \mathbb{R}^n $, showing that their symmetric convex hull is $ m $-centrally neighborly with $ m \sim n / \log^2(cN/n) $ with high probability.