[Paper Review] Non-Gaussian Observations in Nonlinear Compressed Sensing via Stein Discrepancies

Larry Goldstein, Xiaohan Wei|arXiv (Cornell University)|Sep 27, 2016

Sparse and Compressive Sensing Techniques17 references3 citations

TL;DR

This paper introduces Stein discrepancies as a tool to analyze nonlinear compressed sensing under non-Gaussian measurements, enabling tighter performance bounds and relaxed conditions compared to prior work. By measuring deviation from normality, the framework improves signal recovery guarantees for nonlinear models with heavy-tailed or skewed sensing distributions, achieving explicit small constants in error bounds and recovering Gaussian and linear sensing as special cases.

ABSTRACT

Performance guarantees for compression in nonlinear models under non-Gaussian observations can be achieved through the use of distributional characteristics that are sensitive to the distance to normality, and which in particular return the value of zero under Gaussian or linear sensing. The use of these characteristics, or discrepancies, improves some previous results in this area by relaxing conditions and tightening performance bounds. In addition, these characteristics are tractable to compute when Gaussian sensing is corrupted by either additive errors or mixing.

Motivation & Objective

• Address performance guarantees in nonlinear compressed sensing when measurements deviate from Gaussianity.
• Overcome limitations in prior work—such as poor bounds, strong smoothness assumptions, and issues with sparsity—under non-Gaussian sensing.
• Develop a framework sensitive to deviations from normality that generalizes to both nonlinear and linear sensing models.
• Provide explicit, small constants in recovery error bounds, improving upon previous results in terms of tightness and tractability.
• Enable signal reconstruction under general sensing distributions by leveraging distributional characteristics tied to normality deviation.

Proposed method

• Introduce Stein discrepancies as a measure of distance between the sensing distribution and the standard normal, which vanish under Gaussian or linear sensing.
• Use the Stein coefficient and zero-bias coupling to characterize the deviation of the sensing distribution from normality.
• Define a loss function $ L_m( extbf{t}) $ based on empirical observations to estimate the true loss $ L( extbf{t}) $, minimizing it over a compact set $ K $ to recover $ extbf{x} $.
• Apply generic chaining and Gaussian process techniques to bound the supremum of the empirical process indexed by $ extbf{t} eq extbf{x} $, using the Gaussian mean width $ ho( ext{K}) $.
• Employ subgaussian and subexponential tail bounds via $ oldsymbol{ ho}_2 $-norms of the sensing vector components to control concentration of measure.
• Derive high-probability bounds on estimation error by combining Bernstein-type inequalities with tail control of quadratic forms $ rac{1}{m} orm{ extbf{a}_i^ op extbf{t} }^2 $.

Experimental results

Research questions

• RQ1 Can performance guarantees in nonlinear compressed sensing be improved under non-Gaussian measurements by using distributional characteristics sensitive to normality deviation?
• RQ2 How can Stein discrepancies be used to tighten error bounds and reduce dependence on strong smoothness or moment assumptions?
• RQ3 To what extent can the framework recover known results for Gaussian and linear sensing as special cases?
• RQ4 What role does the $ oldsymbol{ ho}_2 $-norm of the sensing vector play in controlling the concentration of the empirical loss?
• RQ5 Can explicit, small constants be derived in recovery error bounds using this discrepancy-based approach?

Key findings

• The proposed framework achieves performance guarantees for nonlinear compressed sensing under non-Gaussian measurements by using Stein discrepancies to measure deviation from normality.
• The method provides tighter error bounds than prior work, with explicit small constants in the recovery error, improving upon previous results in terms of quantitative tightness.
• The framework recovers both Gaussian and linear sensing models as special cases, since Stein discrepancies vanish under these conditions.
• The use of zero-bias coupling and the Stein coefficient enables tractable computation of distributional characteristics even under non-Gaussian or corrupted sensing (e.g., additive noise or mixing).
• High-probability bounds on estimation error are derived using generic chaining and Bernstein’s inequality, with tail control via $ oldsymbol{ ho}_2 $-norms of the sensing vector components.
• The analysis shows that the estimation error $ orm{ extbf{x} - extbf{x}_m }_2 $ is bounded with high probability by a term proportional to $ ho(K) imes ext{Stein discrepancy} imes rac{1}{ ext{sample size}} $, with explicit constants.