[Paper Review] Exact Recovery of Sparse Signals via Orthogonal Matching Pursuit: How Many Iterations Do We Need?

Jian Wang, Byonghyo Shim|arXiv (Cornell University)|Nov 19, 2012

Sparse and Compressive Sensing Techniques33 references9 citations

TL;DR

This paper establishes that orthogonal matching pursuit (OMP) can exactly recover any K-sparse signal within ⌈2.8K⌉ iterations when the measurement matrix satisfies the restricted isometry property (RIP) with δ_{K+1} < 1/√(K+1). The work improves upon prior bounds and closes the gap between theoretical limits and practical performance of OMP, providing a tighter iteration count for guaranteed exact recovery under standard RIP conditions.

ABSTRACT

Orthogonal matching pursuit (OMP) is a greedy algorithm widely used for the recovery of sparse signals from compressed measurements. In this paper, we analyze the number of iterations required for the OMP algorithm to perform exact recovery of sparse signals. Our analysis shows that OMP can accurately recover all $K$-sparse signals within $\lceil 2.8 K ceil$ iterations when the measurement matrix satisfies a restricted isometry property (RIP). Our result improves upon the recent result of Zhang and also bridges the gap between Zhang's result and the fundamental limit of OMP at which exact recovery of $K$-sparse signals cannot be uniformly guaranteed.

Motivation & Objective

• To determine the minimum number of iterations required for OMP to exactly recover K-sparse signals under standard RIP conditions.
• To close the gap between existing theoretical bounds and the fundamental limit of OMP performance in terms of iteration count.
• To improve upon Zhang's recent result showing recovery in 30K iterations under δ_{31K} < 1/3.
• To establish a tighter, more practical upper bound on the number of iterations needed for uniform exact recovery across all K-sparse signals.

Proposed method

• Analyzes OMP's iterative support recovery process under the restricted isometry property (RIP), focusing on residual energy decay and support set evolution.
• Uses a recursive analysis framework to bound the residual norm over successive iterations, leveraging the RIP to control measurement matrix behavior.
• Introduces a novel iterative grouping and bounding technique to track residual energy reduction, using parameters like c, γ, and σ to model decay rates.
• Applies Hermite's identity and mathematical induction to prove key inequalities governing iteration count bounds, particularly in the derivation of (A.24).
• Employs a recursive residual energy bound involving δ_{|T∪T^{k+⌊cγ⌋}|} and exponential decay terms to derive convergence guarantees.
• Combines lemmas on residual energy and RIP to derive a final upper bound on residual norm that ensures exact recovery within a finite number of iterations.

Experimental results

Research questions

• RQ1 What is the minimal number of iterations required for OMP to guarantee exact recovery of all K-sparse signals under standard RIP conditions?
• RQ2 How does the iteration count bound in this work compare to Zhang's result of 30K iterations under δ_{31K} < 1/3?
• RQ3 Can the gap between theoretical limits and practical performance of OMP be closed by tightening the iteration count bound?
• RQ4 What role does the restricted isometry constant δ_{K+1} play in determining the maximum number of iterations needed for exact recovery?

Key findings

• OMP can exactly recover any K-sparse signal within ⌈2.8K⌉ iterations when the measurement matrix satisfies the restricted isometry property with δ_{K+1} < 1/√(K+1).
• This result improves upon Zhang's bound of 30K iterations under δ_{31K} < 1/3, offering a significantly tighter and more practical iteration count.
• The bound of ⌈2.8K⌉ iterations is shown to be nearly optimal, as it approaches the fundamental limit where exact recovery cannot be uniformly guaranteed for K-sparse signals.
• The analysis establishes a residual energy decay rate that ensures convergence to zero within ⌈2.8K⌉ iterations, proving exact recovery is achievable.
• The proof framework, including the use of recursive grouping and induction on (A.24), provides a rigorous foundation for bounding OMP's performance under RIP.
• The result confirms that OMP's performance is robust and predictable under standard RIP conditions, with a clear, tight upper bound on iteration count.