[Paper Review] Online Recovery Guarantees and Analytical Results for OMP

Nazım Burak Karahanoğlu, Hakan Erdoğan|arXiv (Cornell University)|Oct 22, 2012

Sparse and Compressive Sensing Techniques30 references4 citations

TL;DR

This paper presents online recovery guarantees for Orthogonal Matching Pursuit (OMP) beyond the standard K iterations, showing exact recovery of K-sparse signals within 1.5K iterations under a relaxed restricted isometry property (RIP) condition. The method leverages bounds on correct and false support estimates to enable practical, early termination with theoretical assurance, outperforming state-of-the-art guarantees that require up to 30K iterations.

ABSTRACT

Orthogonal Matching Pursuit (OMP) is a simple, yet empirically competitive algorithm for sparse recovery. Recent developments have shown that OMP guarantees exact recovery of K-sparse signals with K or more than K iterations if the observation matrix satisfies the restricted isometry property (RIP) with some conditions. We develop RIP-based online guarantees for recovery of a K-sparse signal with more than K OMP iterations. Though these guarantees cannot be generalized to all sparse signals a priori, we show that they can still hold online when the state-of-the-art K-step recovery guarantees fail. In addition, we present bounds on the number of correct and false indices in the support estimate for the derived condition to be less restrictive than the K-step guarantees. Under these bounds, this condition guarantees exact recovery of a K-sparse signal within 3K/2 iterations, which is much less than the number of steps required for the state-of-the-art exact recovery guarantees with more than K steps. Moreover, we present phase transitions of OMP in comparison to basis pursuit and subspace pursuit, which are obtained after extensive recovery simulations involving different sparse signal types. Finally, we empirically analyse the number of false indices in the support estimate, which indicates that these do not violate the developed upper bound in practice.

Motivation & Objective

• To develop online recovery guarantees for OMP that extend beyond K iterations, enabling earlier termination with theoretical assurance.
• To reduce the number of required iterations compared to state-of-the-art exact recovery guarantees, which demand up to 30K iterations.
• To analyze the number of false indices in support estimation and validate the theoretical bounds empirically.
• To compare OMP's recovery performance with basis pursuit and subspace pursuit using phase transition simulations.
• To establish conditions under which OMP can guarantee exact recovery even when standard K-step guarantees fail.

Proposed method

• Proposes an online recovery condition based on the number of correct (nc) and false (nf) indices in the support estimate at intermediate iterations.
• Derives a sufficient RIP condition involving δK+nf+1 and δK+1 that ensures exact recovery within 1.5K iterations.
• Introduces a theoretical bound requiring nf < ⌈K/2⌉, which is shown to be empirically loose in simulations.
• Employs extensive recovery simulations across different sparse signal types to evaluate phase transitions of OMP, BP, and SP.
• Uses a residue-based termination rule to assess recovery success and failure rates in OMP with more than K iterations.
• Analyzes empirical histograms of failed iterations (nf) to validate the theoretical upper bound and assess practical feasibility.

Experimental results

Research questions

• RQ1 Can OMP achieve exact recovery within 1.5K iterations under a less restrictive RIP condition than existing K-step guarantees?
• RQ2 Under what conditions on correct and false support estimates can online recovery guarantees be established for OMP beyond K iterations?
• RQ3 How does the number of false indices (nf) in OMP's support estimate compare to the theoretical upper bound of ⌈K/2⌉ in practice?
• RQ4 How does OMP's recovery performance compare to basis pursuit and subspace pursuit in terms of phase transitions across different sparse signal distributions?
• RQ5 Can the proposed online recovery condition be applied when standard K-step guarantees fail?

Key findings

• The proposed online recovery condition guarantees exact recovery of K-sparse signals within 1.5K iterations under a relaxed RIP condition, significantly fewer than the 30K iterations required by prior state-of-the-art guarantees.
• The theoretical bound nf < ⌈K/2⌉ on false indices is empirically validated as loose, as OMP e never exceeds K/4 false steps in tested cases.
• Phase transition simulations show OMP e outperforms basis pursuit and subspace pursuit for sparse signals with Gaussian or uniform nonzero coefficients.
• For M=125, K=40 and M=150, K=52, OMP e achieves perfect recovery on all 200 instances where OMP K fails, with no false indices in successful cases.
• The derived condition enables online recovery guarantees even when standard K-step guarantees fail, offering a practical alternative for applications with limited iteration budgets.
• The analysis confirms that the number of false indices does not violate the theoretical upper bound in practice, supporting the validity of the proposed condition.