[Paper Review] Near-Oracle Performance of Basis Pursuit under Random Noise

Zvika Ben‐Haim, Yonina C. Eldar+1 more|arXiv (Cornell University)|Mar 26, 2009

Sparse and Compressive Sensing Techniques16 references13 citations

TL;DR

This paper analyzes the performance of basis pursuit denoising (BPDN), orthogonal matching pursuit (OMP), and thresholding in recovering a sparse vector from noisy, underdetermined measurements. It demonstrates that all three methods achieve near-oracle performance with high probability under mild coherence-based conditions, offering non-asymptotic guarantees applicable to arbitrary dictionaries, with performance differences emerging at varying signal-to-noise ratios.

ABSTRACT

We consider the problem of estimating a deterministic sparse vector x from underdetermined measurements Ax+w, where w represents white Gaussian noise and A is a given deterministic dictionary. We analyze the performance of three sparse estimation algorithms: basis pursuit denoising (BPDN), orthogonal matching pursuit (OMP), and thresholding. These algorithms are shown to achieve near-oracle performance with high probability, assuming that x is sufficiently sparse. Our results are non-asymptotic and are based only on the coherence of A, so that they are applicable to arbitrary dictionaries. Differences in the precise conditions required for the performance guarantees of each algorithm are manifested in the observed performance at high and low signal-to-noise ratios. This provides insight on the advantages and drawbacks of convex relaxation techniques such as BPDN as opposed to greedy approaches such as OMP and thresholding.

Motivation & Objective

• To establish non-asymptotic performance guarantees for sparse recovery algorithms under random white Gaussian noise.
• To compare the robustness and accuracy of convex relaxation (BPDN) versus greedy methods (OMP, thresholding) in noisy, underdetermined systems.
• To quantify how coherence of the measurement dictionary affects recovery performance across different signal-to-noise ratios.
• To provide theoretical insight into the trade-offs between computational complexity and estimation accuracy in sparse recovery.

Proposed method

• The analysis is based on the coherence of the measurement matrix A, defined as the maximum absolute inner product between any two distinct columns.
• Performance bounds are derived using probabilistic arguments that rely solely on the coherence of A, without requiring statistical assumptions on the sparsity pattern.
• The paper evaluates three algorithms: basis pursuit denoising (BPDN), orthogonal matching pursuit (OMP), and hard thresholding, under the same measurement model.
• It establishes high-probability error bounds for each algorithm that scale with the noise level and the sparsity of the signal.
• Theoretical guarantees are non-asymptotic, meaning they hold for finite sample sizes and do not rely on large-n approximations.
• The analysis reveals that BPDN achieves near-oracle performance under weaker conditions than OMP and thresholding, especially at low SNR.

Experimental results

Research questions

• RQ1 Under what conditions do BPDN, OMP, and thresholding achieve near-oracle performance in the presence of white Gaussian noise?
• RQ2 How does the coherence of the measurement dictionary affect the recovery error of sparse estimation algorithms?
• RQ3 What are the relative advantages of convex relaxation (BPDN) versus greedy pursuit (OMP, thresholding) at different signal-to-noise ratios?
• RQ4 Can non-asymptotic performance guarantees be derived for these algorithms using only coherence-based assumptions?
• RQ5 How do the theoretical error bounds of each algorithm compare under identical noise and sparsity conditions?

Key findings

• All three algorithms—BPDN, OMP, and thresholding—achieve near-oracle performance with high probability when the signal is sufficiently sparse and the measurement matrix has low coherence.
• BPDN provides the most robust performance across a wide range of signal-to-noise ratios, particularly excelling at low SNR due to its convex relaxation properties.
• OMP and thresholding show improved performance at high SNR, but their error bounds degrade more rapidly than BPDN's as noise increases.
• Theoretical error bounds for all methods scale with the noise level and the sparsity of the signal, with BPDN achieving the tightest bounds under the same conditions.
• The coherence of the dictionary is the sole determinant of performance guarantees, making the results applicable to arbitrary deterministic dictionaries.
• The results are non-asymptotic, meaning they hold for finite-dimensional problems without requiring large-sample approximations.