[Paper Review] The performance of orthogonal multi-matching pursuit under RIP

Zhiqiang Xu|arXiv (Cornell University)|Oct 19, 2012

Sparse and Compressive Sensing Techniques17 references16 citations

TL;DR

This paper establishes that orthogonal multi-matching pursuit (OMMP) with a fixed parameter $M_0 \leq 8$ can exactly recover any $s$-sparse signal within $s$ iterations when the measurement matrix satisfies $(9s, 1/10)$-RIP. For slowly-decaying signals, OMMP with $M = s^a$ ($a \in [0, 1/2]$) achieves recovery in $O(s^{1-a})$ iterations, significantly reducing computational complexity compared to OMP.

ABSTRACT

The orthogonal multi-matching pursuit (OMMP) is a natural extension of orthogonal matching pursuit (OMP). We denote the OMMP with the parameter $M$ as OMMP(M) where $M\geq 1$ is an integer. The main difference between OMP and OMMP(M) is that OMMP(M) selects $M$ atoms per iteration, while OMP only adds one atom to the optimal atom set. In this paper, we study the performance of orthogonal multi-matching pursuit (OMMP) under RIP. In particular, we show that, when the measurement matrix A satisfies $(9s, 1/10)$-RIP, there exists an absolutely constant $M_0\leq 8$ so that OMMP(M_0) can recover $s$-sparse signal within $s$ iterations. We furthermore prove that, for slowly-decaying $s$-sparse signal, OMMP(M) can recover s-sparse signal within $O(\frac{s}{M})$ iterations for a large class of $M$. In particular, for $M=s^a$ with $a\in [0,1/2]$, OMMP(M) can recover slowly-decaying $s$-sparse signal within $O(s^{1-a})$ iterations. The result implies that OMMP can reduce the computational complexity heavily.

Motivation & Objective

• To determine whether there exists a fixed $M_0$ such that OMMP($M_0$) can recover all $s$-sparse signals within exactly $s$ iterations under a bounded RIP constant.
• To analyze the trade-off between the number of iterations and the selection parameter $M$ in OMMP for slowly-decaying $s$-sparse signals.
• To establish theoretical guarantees for OMMP under a more natural RIP condition with an absolute constant RIP parameter, rather than one scaling with $s$.

Proposed method

• The analysis uses restricted isometry property (RIP) with a fixed constant $\delta = 1/10$ for $9s$-sparse signals to bound the error reduction per iteration.
• A key inequality is derived to relate the residual norm reduction to the energy of the remaining signal components, leveraging the RIP and the decay property of the signal.
• An inductive argument is applied to show that the support of the estimated signal $\Lambda^\ell$ remains within the true support of the $s$-sparse signal $\mathbf{x}$, provided the signal is $\alpha$-decaying with $\alpha > \sqrt{(1+\delta_s)/(2 - (1+\delta_s)^2)}$.
• The proof relies on bounding the residual norm $\|\mathbf{y} - A\mathbf{x}^\ell\|_2^2$ and showing it decreases geometrically when the correct atoms are selected.
• The algorithm selects $M$ atoms per iteration based on the largest correlation with the current residual, then performs a least-squares update on the selected support.
• Spark and linear independence conditions are used to ensure that the least-squares solution on the selected support recovers the true signal exactly when the support is correct.

Experimental results

Research questions

• RQ1 Does there exist an absolute constant $M_0 \leq 8$ such that OMMP($M_0$) can recover all $s$-sparse signals within exactly $s$ iterations under $(9s, 1/10)$-RIP?
• RQ2 For $M = s^a$ with $a \in [0, 1/2]$, can OMMP recover slowly-decaying $s$-sparse signals within $O(s^{1-a})$ iterations?
• RQ3 What is the minimal number of iterations required for OMMP to recover $s$-sparse signals when the RIP constant is an absolute constant rather than scaling with $s$?

Key findings

• OMMP($M_0$) with $M_0 \leq 8$ can recover any $s$-sparse signal within $s$ iterations when the measurement matrix satisfies $(9s, 1/10)$-RIP.
• For slowly-decaying $s$-sparse signals, OMMP with $M = s^a$ for $a \in [0, 1/2]$ recovers the signal in $O(s^{1-a})$ iterations.
• The residual error decreases geometrically when the signal is $\alpha$-decaying with $\alpha > \sqrt{(1+\delta_s)/(2 - (1+\delta_s)^2)}$, ensuring convergence to the true support.
• The spark of the matrix $A$ must exceed $8s\log_2(2(s+1))$ to guarantee exact recovery via least-squares on the selected support.
• The algorithm maintains support inclusion throughout iterations under the given RIP and decay conditions, ensuring no incorrect atoms are added.
• The theoretical analysis shows that OMMP achieves significant computational savings by reducing the number of iterations from $s$ to $O(s^{1-a})$ for $M = s^a$, especially when $a$ is small.