[Paper Review] Random Sampling of Sparse Trigonometric Polynomials II - Orthogonal Matching Pursuit versus Basis Pursuit

Stefan Kunis, Holger Rauhut|ArXiv.org|Apr 19, 2006

Sparse and Compressive Sensing Techniques35 references4 citations

TL;DR

This paper compares Basis Pursuit (BP) and greedy algorithms—Orthogonal Matching Pursuit (OMP) and Thresholding—for reconstructing sparse multivariate trigonometric polynomials from few random samples. It establishes theoretical success probabilities for OMP and Thresholding under continuous and discrete sampling models, showing that OMP achieves similar recovery rates to BP but with significantly faster computation, making it preferable in practice despite a non-uniform recovery guarantee.

ABSTRACT

We investigate the problem of reconstructing sparse multivariate trigonometric polynomials from few randomly taken samples by Basis Pursuit and greedy algorithms such as Orthogonal Matching Pursuit (OMP) and Thresholding. While recovery by Basis Pursuit has recently been studied by several authors, we provide theoretical results on the success probability of reconstruction via Thresholding and OMP for both a continuous and a discrete probability model for the sampling points. We present numerical experiments, which indicate that usually Basis Pursuit is significantly slower than greedy algorithms, while the recovery rates are very similar.

Motivation & Objective

• To analyze the theoretical success probability of Orthogonal Matching Pursuit (OMP) and Thresholding in recovering sparse trigonometric polynomials from random samples.
• To compare the performance of OMP, Thresholding, and Basis Pursuit (BP) in terms of recovery success rate and computational efficiency.
• To investigate whether greedy algorithms like OMP can achieve uniform recovery guarantees comparable to BP, despite their non-uniform nature.
• To evaluate the stability of OMP under noise and its ability to recover high-frequency or closely spaced components.
• To assess the practical feasibility of OMP for real-world applications where speed and accuracy are critical.

Proposed method

• Theoretical analysis of OMP and Thresholding using the coherence parameter of random Fourier matrices to derive success probability bounds.
• Adaptation of the restricted isometry property (RIP) framework to random sampling points in the continuous cube $[0,2 heta]^d$ and on discrete grids.
• Implementation of OMP using the nonequispaced fast Fourier transform (NFFT) and fast least-squares solvers (LSQR) for efficiency.
• Use of Basis Pursuit via convex optimization solvers (MOSEK, CVX, L1MAGIC) for comparison, with FFT and NFFT acceleration.
• Numerical experiments on synthetic signals with varying dimensions $D$, sparsity $M$, and sample counts $N$, using both uniform and non-uniform sampling.
• Noise robustness evaluation by corrupting samples with Gaussian noise and measuring coefficient recovery error and support set accuracy.

Experimental results

Research questions

• RQ1 What is the theoretical success probability of OMP and Thresholding in reconstructing sparse trigonometric polynomials from random samples?
• RQ2 How do the recovery rates of OMP and BP compare under similar sampling conditions and sparsity levels?
• RQ3 Can OMP achieve uniform recovery guarantees across all sparse trigonometric polynomials, or is its success limited to specific signals?
• RQ4 How does the computational complexity of OMP compare to BP, especially when leveraging fast transforms like FFT and NFFT?
• RQ5 Is OMP stable under noise, and can it accurately recover the true support and coefficients when samples are corrupted?

Key findings

• OMP achieves recovery success rates comparable to Basis Pursuit (BP) on generic signals, even at low oversampling ratios $N/M$.
• Despite a non-uniform recovery guarantee, OMP outperforms Thresholding and matches BP in practice, especially in terms of support recovery accuracy.
• OMP is significantly faster than BP—especially when using fast solvers like LSQR and NFFT—making it more suitable for large-scale problems.
• Numerical experiments confirm that OMP can recover signals with up to 10 frequencies in the range $\{-2^{18}, \dots, 2^{18}-1\}$ from only 60 random samples.
• OMP demonstrates stability under moderate noise, correctly identifying the true support set and producing small coefficient errors even when SNR is around 15.6 dB.
• Theoretical analysis shows that the coherence-based success probability for OMP’s first step is non-trivial, though full convergence analysis remains open due to stochastic dependencies.