[Paper Review] Robustness to unknown error in sparse regularization

Simone Brugiapaglia, Ben Adcock|arXiv (Cornell University)|May 29, 2017

Sparse and Compressive Sensing Techniques30 references3 citations

TL;DR

This paper establishes robustness guarantees for quadratically-constrained basis pursuit (QCBP) in compressed sensing without requiring a priori knowledge of noise levels. By introducing an abstract framework based on quotients and robust instance optimality, the authors prove that QCBP remains stable and robust under unknown errors for random Gaussian matrices and heavy-tailed matrices, including those from bounded orthonormal systems, with explicit error bounds scaling as $\mathcal{O}(\sqrt{m/N})$. This resolves a key theoretical gap between practice and existing analysis.

ABSTRACT

Quadratically-constrained basis pursuit has become a popular device in sparse regularization; in particular, in the context of compressed sensing. However, the majority of theoretical error estimates for this regularizer assume an a priori bound on the noise level, which is usually lacking in practice. In this paper, we develop stability and robustness estimates which remove this assumption. First, we introduce an abstract framework and show that robust instance optimality of any decoder in the noise-aware setting implies stability and robustness in the noise-blind setting. This is based on certain sup-inf constants referred to as quotients, strictly related to the quotient property of compressed sensing. We then apply this theory to prove the robustness of quadratically-constrained basis pursuit under unknown error in the cases of random Gaussian matrices and of random matrices with heavy-tailed rows, such as random sampling matrices from bounded orthonormal systems. We illustrate our results in several cases of practical importance, including subsampled Fourier measurements and recovery of sparse polynomial expansions.

Motivation & Objective

• To close the theoretical gap between the practical robustness of QCBP and existing analyses that assume known noise levels.
• To develop a general framework for analyzing stability and robustness of sparse recovery in the absence of a priori error bounds.
• To establish quantitative error bounds for QCBP under unknown error in random Gaussian and heavy-tailed sensing matrices.
• To validate the theory in practical settings such as subsampled Fourier measurements and sparse polynomial recovery.

Proposed method

• Introduces an abstract framework linking robust instance optimality in the noise-aware setting to stability and robustness in the noise-blind setting via sup-inf constants (quotients).
• Defines and analyzes the quotient property, which generalizes the restricted isometry property and is central to the robustness analysis.
• Applies the framework to random Gaussian matrices and matrices with heavy-tailed rows, such as those from bounded orthonormal systems.
• Uses concentration inequalities and tail bounds to control the deviation of the normalized Christoffel function from 1, ensuring bounded distortion.
• Derives explicit error bounds for QCBP under unknown error, showing decay as $\mathcal{O}(\sqrt{m/N})$ for $m \ll N$.
• Employs union bounds and arcsin-based estimates to control the probability of large deviations in row norms of random matrices.

Experimental results

Research questions

• RQ1 Can robustness of QCBP be guaranteed without assuming a known bound on the noise level in compressed sensing?
• RQ2 How do quotients and the quotient property relate to the stability of sparse recovery in the noise-blind setting?
• RQ3 What error bounds can be derived for QCBP when the sensing matrix has heavy-tailed rows, such as in Fourier sampling?
• RQ4 How does the performance of QCBP scale with the number of measurements $m$ and signal dimension $N$ under unknown error?
• RQ5 Can the theoretical framework be applied to practical cases like subsampled Fourier measurements and sparse polynomial expansions?

Key findings

• The paper proves that robust instance optimality in the noise-aware setting implies stability and robustness in the noise-blind setting via quotient-based analysis.
• For random Gaussian matrices, QCBP achieves robust recovery with error bounds scaling as $\mathcal{O}(\sqrt{m/N})$ under unknown error.
• For matrices with heavy-tailed rows, such as those from bounded orthonormal systems, the same $\mathcal{O}(\sqrt{m/N})$ error decay is established.
• Explicit bounds are derived for the distortion term $\xi$, showing $\xi \leq 9m\sqrt{\varepsilon/\pi}$ under appropriate conditions on $N$ and $m$.
• The framework applies to practical scenarios including partial Fourier measurements and sparse polynomial recovery, with numerical results confirming robustness even when $\|\mathbf{e}\|_2$ is unknown.
• The analysis shows that the recovery error remains bounded and decays as $m$ increases, even when the noise level is not known in advance.