[Paper Review] Recovery of Sparse Signals Using Multiple Orthogonal Least Squares

Jian Wang, Ping Li|arXiv (Cornell University)|Oct 9, 2014

Sparse and Compressive Sensing Techniques42 references3 citations

TL;DR

This paper proposes Multiple Orthogonal Least Squares (MOLS), a greedy sparse recovery algorithm that selects multiple support indices per iteration to accelerate convergence and improve computational efficiency. Theoretically, MOLS guarantees exact recovery of $K$-sparse signals within $K$ iterations if the measurement matrix satisfies restricted isometry property (RIP) with $\delta_{LK} < \frac{\sqrt{L}}{\sqrt{K}+2\sqrt{L}}$, and stable recovery is achievable under linear SNR scaling with sparsity.

ABSTRACT

We study the problem of recovering sparse signals from compressed linear measurements. This problem, often referred to as sparse recovery or sparse reconstruction, has generated a great deal of interest in recent years. To recover the sparse signals, we propose a new method called multiple orthogonal least squares (MOLS), which extends the well-known orthogonal least squares (OLS) algorithm by allowing multiple $L$ indices to be chosen per iteration. Owing to inclusion of multiple support indices in each selection, the MOLS algorithm converges in much fewer iterations and improves the computational efficiency over the conventional OLS algorithm. Theoretical analysis shows that MOLS ($L &gt; 1$) performs exact recovery of all $K$-sparse signals within $K$ iterations if the measurement matrix satisfies the restricted isometry property (RIP) with isometry constant $δ_{LK} &lt; \frac{\sqrt{L}}{\sqrt{K} + 2 \sqrt{L}}.$ The recovery performance of MOLS in the noisy scenario is also studied. It is shown that stable recovery of sparse signals can be achieved with the MOLS algorithm when the signal-to-noise ratio (SNR) scales linearly with the sparsity level of input signals.

Motivation & Objective

• To address the computational inefficiency of conventional OLS by accelerating convergence through multi-index selection.
• To improve recovery accuracy and reduce iteration count in sparse signal reconstruction from compressed measurements.
• To establish theoretical conditions under which MOLS achieves exact and stable recovery of sparse signals.
• To extend the OLS framework by allowing simultaneous selection of $L > 1$ indices per iteration, enhancing performance over OMP and OLS.

Proposed method

• MOLS selects $L$ indices per iteration by minimizing the residual power over all possible $L$-element extensions of the current support set.
• At each iteration, the algorithm identifies the set $\mathcal{S}^k$ of $L$ indices that most reduce the residual energy when added to the current support $\mathcal{T}^{k-1}$.
• The signal estimate $\mathbf{x}^k$ is computed via least-squares projection onto the updated support $\mathcal{T}^k$, ensuring optimal coefficient estimation at each step.
• The residual is updated as $\mathbf{r}^k = \mathbf{y} - \mathbf{\Phi} \mathbf{x}^k$, and the process continues until convergence or $K$ iterations.
• Theoretical analysis leverages the restricted isometry property (RIP) to bound residual reduction and derive recovery conditions.
• A key component is the use of projection-based residual minimization, which ensures orthogonal updates and faster convergence than OMP or OLS.

Experimental results

Research questions

• RQ1 Can selecting multiple indices per iteration in OLS significantly reduce the number of iterations required for exact sparse signal recovery?
• RQ2 What is the theoretical condition on the measurement matrix (in terms of RIP) that guarantees exact recovery using MOLS?
• RQ3 How does the signal-to-noise ratio (SNR) affect the stable recovery performance of MOLS in noisy settings?
• RQ4 Does MOLS outperform OLS and OMP in terms of computational efficiency and recovery accuracy under the same conditions?

Key findings

• MOLS achieves exact recovery of all $K$-sparse signals within $K$ iterations if the measurement matrix satisfies $\delta_{LK} < \frac{\sqrt{L}}{\sqrt{K}+2\sqrt{L}}$.
• The algorithm converges in significantly fewer iterations than OLS due to the selection of $L > 1$ indices per iteration, improving computational efficiency.
• In noisy environments, stable recovery is achieved when the SNR scales linearly with the sparsity level $K$ of the input signal.
• Theoretical analysis confirms that MOLS maintains strong convergence properties by minimizing residual energy through orthogonal projection at each step.
• MOLS outperforms OMP and OLS in both convergence speed and recovery accuracy, particularly in high-sparsity regimes.
• The method is robust to noise, with recovery error bounded under appropriate SNR conditions, ensuring reliable performance in practical applications.