[Paper Review] A Study on Sparse Vector Distributions and Recovery from Compressed Sensing

Bob L. Sturm|arXiv (Cornell University)|Mar 31, 2011

Sparse and Compressive Sensing TechniquesEngineering23 references9 citations

TL;DR

This study evaluates recovery algorithms for compressed sensing under varying sparse vector distributions, finding that orthogonal matching pursuit (OMP) outperforms 1-minimization and two-stage thresholding for Normal and Laplacian distributions. Selecting the best solution across multiple algorithms significantly boosts exact recovery probability.

ABSTRACT

I empirically investigate the variability of several recovery algorithms on the distribution underlying the sparse vector sensed by a random matrix. a dependence that has been noted before, but, to my knowledge, not thoroughly investigated. I find that ‘1-minimization [1] and tuned two-stage thresholding [2] (subspace pursuit [3] without the use of a sparsity oracle) are the most robust to changes in the sparse vector distribution; but they are outperformed to a large degree by greedy methods, such as orthogonal matching pursuit [4] for sparse vectors distributed Normal and Laplacian. I also find that selecting the best solution from those produced by several recovery algorithms can significantly increase the probability of exact recovery.

Motivation & Objective

• To investigate how sparse vector distributions affect the performance of compressed sensing recovery algorithms.
• To assess the robustness of different recovery algorithms—such as 1-minimization and greedy methods—under distributional variability.
• To determine whether combining outputs from multiple algorithms enhances exact recovery probability.
• To provide empirical evidence on algorithmic performance beyond idealized assumptions about sparse vector structure.

Proposed method

• Empirically evaluate multiple recovery algorithms using random sensing matrices across different sparse vector distributions.
• Apply 1-minimization and tuned two-stage thresholding (subspace pursuit without sparsity oracle) as baseline methods.
• Use orthogonal matching pursuit (OMP) as a representative greedy method for comparison.
• Compare recovery performance across distributions, focusing on exact recovery rates.
• Implement ensemble selection by choosing the best solution from multiple algorithm outputs.
• Measure performance using exact recovery probability as the primary metric across varied distributions.

Experimental results

Research questions

• RQ1 How does the distribution of the sparse vector affect the recovery performance of 1-minimization and two-stage thresholding?
• RQ2 How do greedy methods like orthogonal matching pursuit compare to convex and thresholding-based methods under different sparse vector distributions?
• RQ3 To what extent does combining outputs from multiple recovery algorithms improve exact recovery probability?
• RQ4 Which recovery algorithm is most robust across diverse sparse vector distributions?

Key findings

• 1-minimization and tuned two-stage thresholding (subspace pursuit without sparsity oracle) show strong robustness to changes in sparse vector distribution.
• Orthogonal matching pursuit (OMP) significantly outperforms 1-minimization and two-stage thresholding for sparse vectors with Normal and Laplacian distributions.
• The performance gap between OMP and other methods is substantial under Normal and Laplacian distributions.
• Selecting the best solution from multiple recovery algorithms leads to a notable increase in the probability of exact recovery.
• The study confirms that distributional assumptions on sparse vectors critically influence algorithmic performance in compressed sensing.
• No single algorithm dominates across all distributions, but OMP is most effective for the commonly used Normal and Laplacian cases.