[Paper Review] Stochastic Recovery Of Sparse Signals From Random Measurements

M. Andrecut|arXiv (Cornell University)|Apr 7, 2013

Sparse and Compressive Sensing Techniques9 references3 citations

TL;DR

This paper proposes a stochastic optimization (SO) method for sparse signal recovery from random measurements, using a weighted ℓ₁ norm derived from spectral entropy to guide search toward non-zero signal components. The method achieves accurate recovery with low computational complexity, outperforming basis pursuit in implementation ease while maintaining competitive reconstruction accuracy, especially at low sparsity levels.

ABSTRACT

Sparse signal recovery from a small number of random measurements is a well known NP-hard to solve combinatorial optimization problem, with important applications in signal and image processing. The standard approach to the sparse signal recovery problem is based on the basis pursuit method. This approach requires the solution of a large convex optimization problem, and therefore suffers from high computational complexity. Here, we discuss a stochastic optimization method, as a low-complexity alternative to the basis pursuit approach.

Motivation & Objective

• Address the high computational complexity of basis pursuit (BP) in compressive sensing by proposing a low-complexity alternative.
• Overcome the NP-hard nature of ℓ₀ minimization in sparse signal recovery through a stochastic optimization framework.
• Improve signal reconstruction by incorporating spectral entropy into the objective function to prioritize non-zero components.
• Enable efficient recovery by reducing the search space to the null subspace of the measurement matrix.
• Demonstrate the feasibility of approximate, yet accurate, signal recovery with minimal computational overhead.

Proposed method

• Formulate the objective function as the product of the ℓ₁ norm and spectral entropy of the candidate solution, equivalent to a weighted ℓ₁ norm.
• Use threshold accepting, a stochastic local search algorithm, to iteratively perturb the solution while maintaining feasibility to the linear constraint system.
• Initialize the solution using the Moore-Penrose pseudoinverse of the measurement matrix: $ x riangleq ilde{oldsymbol{ m extbackslash Phi}}^{ op} y $.
• Apply a cooling schedule with decreasing temperature $ heta $ and step size $ eta $, where $ heta_{ ext{new}} = heta imes ho $, $ ho = ( heta_f / heta_i)^{1/T} $.
• At each iteration, perturb each component of the solution using random sign flips scaled by the null space projection vector $ oldsymbol{q}^{(m)} $, and accept improvements based on the thresholded objective function.
• Ensure all candidate solutions satisfy $ ilde{oldsymbol{ m extbackslash Phi}}^T ilde{x} = y $, restricting the search to the null subspace of dimension $ R = M - N $.

Experimental results

Research questions

• RQ1 Can a stochastic optimization method achieve competitive sparse signal recovery performance with significantly reduced computational complexity compared to basis pursuit?
• RQ2 How does incorporating spectral entropy into the objective function improve convergence and reconstruction accuracy?
• RQ3 What is the impact of sparsity level $ K $ and number of measurements $ N $ on the recovery error in the proposed stochastic framework?
• RQ4 To what extent does the method maintain feasibility of the solution throughout the optimization process?
• RQ5 Does the proposed method exhibit a phase transition in recovery error similar to that observed in basis pursuit, depending on the $ K/N $ ratio?

Key findings

• For $ M = 100 $, $ N = 50 $, and $ K = 20 $, the recovery error was $ 8.532 \times 10^{-4}\% $, indicating near-perfect reconstruction at low sparsity.
• As sparsity increased to $ K = 25 $, the error rose to $ 3.145\% $, and to $ 11.329\% $ at $ K = 30 $, demonstrating a clear phase transition in performance.
• The method achieved an average recovery error below 10% across a large region of the $ K $-$ N $ parameter space, as shown in the phase transition plot (Figure 3).
• Contour lines of constant error showed logarithmic dependence on $ K $ and $ N $, similar to those observed in basis pursuit, indicating comparable theoretical behavior.
• The algorithm reduced the search space from $ \mathbb{R}^M $ to $ \mathbb{R}^R $, where $ R = M - N $, significantly improving efficiency by constraining updates to the null subspace.
• The objective function's weighting via self-information (from spectral entropy) effectively guided the search toward non-zero components, enhancing convergence and accuracy.