[Paper Review] A Simple Message-Passing Algorithm for Compressed Sensing

Venkat Chandar, Devavrat Shah+1 more|arXiv (Cornell University)|Jan 23, 2010

Sparse and Compressive Sensing Techniques17 references4 citations

TL;DR

This paper proposes a simple message-passing algorithm for compressed sensing that recovers nonnegative sparse vectors from binary measurement matrices derived from bipartite expander graphs. It guarantees exact recovery of k-sparse signals in O(n log(n/k) log k) time using O(k log(n/k)) measurements, and provides an ℓ₁/ℓ₁ error bound for general nonnegative signals under mild expansion conditions.

ABSTRACT

We consider the recovery of a nonnegative vector x from measurements y = Ax, where A is an m-by-n matrix whos entries are in {0, 1}. We establish that when A corresponds to the adjacency matrix of a bipartite graph with sufficient expansion, a simple message-passing algorithm produces an estimate \hat{x} of x satisfying ||x-\hat{x}||_1 \leq O(n/k) ||x-x(k)||_1, where x(k) is the best k-sparse approximation of x. The algorithm performs O(n (log(n/k))^2 log(k)) computation in total, and the number of measurements required is m = O(k log(n/k)). In the special case when x is k-sparse, the algorithm recovers x exactly in time O(n log(n/k) log(k)). Ultimately, this work is a further step in the direction of more formally developing the broader role of message-passing algorithms in solving compressed sensing problems.

Motivation & Objective

• To develop a deterministic, low-complexity message-passing algorithm for compressed sensing of nonnegative vectors.
• To establish theoretical guarantees for exact and approximate recovery using only binary measurement matrices derived from bipartite expanders.
• To improve upon existing combinatorial algorithms by reducing required expansion factor and running time while maintaining strong error bounds.
• To extend message-passing analysis from coding theory to compressed sensing with adversarial error models.
• To demonstrate that message-passing can achieve near-optimal performance with provable guarantees in the compressed sensing framework.

Proposed method

• The algorithm uses a message-passing framework on a bipartite graph where the measurement matrix A is the adjacency matrix of the graph.
• Messages are passed iteratively between variable nodes (signal components) and check nodes (measurements), updating estimates of signal values based on local neighborhood information.
• The algorithm leverages a (1/2 + ε)-matching construction in the graph to ensure sufficient connectivity and error correction capability.
• The analysis uses a recursive error bound based on the ℓ₁ norm of the estimation error, tracking the evolution of error over iterations.
• A max-flow formulation is used to prove the existence of a (1/2 + ε)-matching in graphs with sufficient expansion, which underpins the convergence guarantee.
• The algorithm performs O(log k) iterations to reduce error, with each iteration processing all nodes in parallel, enabling efficient implementation.

Experimental results

Research questions

• RQ1 Can a simple message-passing algorithm achieve exact recovery of k-sparse nonnegative signals using only O(k log(n/k)) measurements?
• RQ2 What expansion properties of the bipartite graph are sufficient to guarantee exact recovery via message-passing?
• RQ3 Can the algorithm provide a bounded ℓ₁/ℓ₁ error guarantee for general nonnegative signals that are not exactly k-sparse?
• RQ4 How does the running time of the message-passing algorithm compare to existing combinatorial and convex optimization-based methods?
• RQ5 Can adversarial error models be handled with message-passing in compressed sensing, as in coding theory?

Key findings

• The algorithm exactly recovers any k-sparse nonnegative vector in O(n log(n/k) log k) time when the measurement matrix corresponds to a bipartite graph with expansion factor greater than 0.5.
• For general nonnegative signals, the algorithm achieves an ℓ₁/ℓ₁ error guarantee: ||x − x̂||₁ ≤ O(n/k) ||x − x⁽ᵏ⁾||₁, where x⁽ᵏ⁾ is the best k-sparse approximation of x.
• The number of measurements required is O(k log(n/k)), matching the information-theoretic lower bound up to logarithmic factors.
• The total computation cost is O(n (log(n/k))² log k), which is faster than most existing combinatorial algorithms in the regime where k scales sublinearly with n.
• The algorithm requires a lower expansion factor (0.5) than the Sipser-Spielman algorithm (0.75), though it only applies to nonnegative signals.
• The analysis provides deterministic guarantees under adversarial noise, unlike prior probabilistic analyses, and is applicable to any nonnegative signal, not just random or sparse ones.