[Paper Review] Sampling numbers of smoothness classes via $\ell^1$-minimization

Thomas Jahn, Tino Ullrich+1 more|arXiv (Cornell University)|Dec 1, 2022

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper establishes new upper bounds for sampling numbers of smoothness classes in $L^2$ using $β^1$-minimization (basis pursuit denoising) with function values. It shows that nonlinear recovery via $β^1$-minimization achieves significantly faster convergence rates than linear methods—up to $m^{-1/2}$ gain in weighted Wiener spaces and arbitrary polynomial speed-up in quasi-Banach cases—especially for mixed-smoothness Sobolev spaces on the $d$-torus when $1 < p < 2$ and $d$ is large.

ABSTRACT

Using techniques developed recently in the field of compressed sensing we prove new upper bounds for general (nonlinear) sampling numbers of (quasi-)Banach smoothness spaces in $L^2$. In particular, we show that in relevant cases such as mixed and isotropic weighted Wiener classes or Sobolev spaces with mixed smoothness, sampling numbers in $L^2$ can be upper bounded by best $n$-term trigonometric widths in $L^\infty$. We describe a recovery procedure from $m$ function values based on $\ell^1$-minimization (basis pursuit denoising). With this method, a significant gain in the rate of convergence compared to recently developed linear recovery methods is achieved. In this deterministic worst-case setting we see an additional speed-up of $m^{-1/2}$ (up to log factors) compared to linear methods in case of weighted Wiener spaces. For their quasi-Banach counterparts even arbitrary polynomial speed-up is possible. Surprisingly, our approach allows to recover mixed smoothness Sobolev functions belonging to $S^r_pW(\mathbb{T}^d)$ on the $d$-torus with a logarithmically better rate of convergence than any linear method can achieve when $1 &lt; p &lt; 2$ and $d$ is large. This effect is not present for isotropic Sobolev spaces.

Motivation & Objective

• To close the gap between linear and nonlinear sampling recovery by developing a constructive, implementable method for multivariate function recovery.
• To analyze the performance of $β^1$-minimization in the context of sampling numbers for smoothness classes in $L^2$.
• To establish quantitative bounds linking $L^2$ recovery error to best $n$-term trigonometric approximation in $L^∞$.
• To demonstrate that nonlinear recovery via $β^1$-minimization can outperform linear methods, especially in high-dimensional mixed-smoothness settings.
• To provide a deterministic, constructive recovery procedure with provable convergence rates, avoiding non-constructive subsampling strategies.

Proposed method

• Uses $β^1$-minimization (basis pursuit denoising) as a recovery operator based on $m$ function values.
• Applies recent restricted isometry property (RIP) results for bounded orthonormal systems to ensure stable recovery.
• Relies on noise-aware guarantees from compressed sensing to control the recovery error in $L^2$.
• Links the $L^2$ recovery error to the best $n$-term approximation error in $L^∞$ via the multivariate trigonometric system.
• Employs a parameter choice in the basis pursuit denoising program to ensure near-optimal $n$-term approximation recovery.
• Derives a sampling number bound involving $m \approx C d \log(d+1) n \log(n+1)^2 \log(M)$ measurements, with error bounded by $\widetilde{C}(\sigma_n(\mathcal{F};\mathcal{T}^d)_{L^\infty} + E_{[-M,M]^d \cap \mathbb{Z}^d}(\mathcal{F};\mathcal{T}^d)_{L^\infty})$.

Experimental results

Research questions

• RQ1 Can $β^1$-minimization achieve better convergence rates than linear sampling recovery for smoothness classes in $L^2$?
• RQ2 What is the relationship between sampling numbers in $L^2$ and best $n$-term trigonometric approximation in $L^\infty$?
• RQ3 Does nonlinear recovery via $β^1$-minimization yield a speed-up over linear methods in high-dimensional mixed-smoothness settings?
• RQ4 Can the proposed method achieve arbitrary polynomial speed-up in quasi-Banach spaces?
• RQ5 Is the recovery procedure constructive and implementable, avoiding non-constructive subsampling strategies?

Key findings

• The sampling number $\varrho_m(\mathcal{F})_{L^2}$ is upper bounded by a constant multiple of the best $n$-term $L^\infty$ trigonometric approximation error $\sigma_n(\mathcal{F};\mathcal{T}^d)_{L^\infty}$, with $m \approx C d \log(d+1) n \log(n+1)^2 \log(M)$ measurements.
• For weighted Wiener spaces, the method achieves an additional $m^{-1/2}$ gain in convergence rate compared to linear methods, up to logarithmic factors.
• In quasi-Banach counterparts of weighted Wiener spaces, arbitrary polynomial speed-up over linear methods is possible.
• For mixed-smoothness Sobolev spaces $S^r_p(\mathbb{T}^d)$ with $1 < p < 2$ and large $d$, the method achieves a logarithmically better convergence rate than any linear method.
• The gain in convergence rate is not present for isotropic Sobolev spaces, highlighting the role of mixed smoothness and dimensionality.
• The recovery method is constructive and avoids non-constructive subsampling strategies, relying instead on convex optimization with provable guarantees.