[Paper Review] Bayesian Compressive Sensing via Belief Propagation

Dror Baron, Shriram Sarvotham+1 more|ArXiv.org|Dec 25, 2008

Sparse and Compressive Sensing Techniques50 references4 citations

TL;DR

This paper proposes Bayesian Compressive Sensing via Belief Propagation (CS-BP), a fast, scalable method for reconstructing sparse signals from sub-Nyquist measurements using approximate Bayesian inference on a factor graph representation. It achieves $O(K\log N)$ measurements and $O(N\log^2 N)$ computation for a length-$N$ signal with $K$ significant coefficients, significantly outperforming conventional $\ell_1$-based methods in speed while maintaining high reconstruction accuracy under a two-state Gaussian mixture model.

ABSTRACT

Compressive sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable, sub-Nyquist signal acquisition. When a statistical characterization of the signal is available, Bayesian inference can complement conventional CS methods based on linear programming or greedy algorithms. We perform approximate Bayesian inference using belief propagation (BP) decoding, which represents the CS encoding matrix as a graphical model. Fast computation is obtained by reducing the size of the graphical model with sparse encoding matrices. To decode a length-N signal containing K large coefficients, our CS-BP decoding algorithm uses O(Klog(N)) measurements and O(Nlog^2(N)) computation. Finally, although we focus on a two-state mixture Gaussian model, CS-BP is easily adapted to other signal models.

Motivation & Objective

• To develop a fast, scalable algorithm for reconstructing sparse signals from compressed linear measurements using Bayesian inference.
• To reduce the computational complexity of existing compressive sensing methods, particularly those based on $\ell_1$ minimization or greedy algorithms.
• To leverage graphical models and belief propagation to perform efficient approximate Bayesian inference on sparse signals.
• To demonstrate that belief propagation decoding with sparse encoding matrices achieves near-optimal performance with significantly reduced computation.
• To extend the applicability of compressive sensing to large-scale problems by enabling faster decoding without sacrificing reconstruction fidelity.

Proposed method

• The method models the compressive sensing problem as a factor graph, representing the measurement matrix $\Phi$ as a graphical model to enable belief propagation (BP) decoding.
• It employs a two-state Gaussian mixture prior on signal coefficients to model sparsity, with large coefficients drawn from $\mathcal{N}(0, \sigma_1^2)$ and small ones from $\mathcal{N}(0, \sigma_0^2)$.
• Belief propagation is used to compute approximate posterior marginals of the signal coefficients, enabling efficient Bayesian inference without solving NP-hard $\ell_0$ optimization.
• The size of the graphical model is reduced by using sparse encoding matrices $\Phi$, where each column has $L = \Theta(\log N)$ non-zero entries, enabling fast message passing.
• The algorithm achieves $O(N\log^2 N)$ computation time by exploiting sparsity in $\Phi$ and efficient message-passing updates on the factor graph.
• The method is adaptable to other signal models beyond the two-state Gaussian mixture, enabling flexible application to various compressible or approximately sparse signals.

Experimental results

Research questions

• RQ1 Can belief propagation be effectively used to perform approximate Bayesian inference in compressive sensing with reduced computational complexity?
• RQ2 What is the minimum number of measurements required for reliable signal reconstruction using BP-based decoding under a Gaussian mixture model?
• RQ3 How does the computational complexity of BP-based decoding scale with signal dimension $N$ and sparsity $K$?
• RQ4 Can sparse encoding matrices preserve reconstruction accuracy while enabling fast message-passing on the factor graph?
• RQ5 How does the performance of CS-BP compare to conventional $\ell_1$-minimization and greedy algorithms in terms of speed and accuracy?

Key findings

• The CS-BP algorithm achieves $O(K\log N)$ measurements, matching the information-theoretic lower bound for sparse signal recovery.
• The algorithm performs $O(N\log^2 N)$ computation, significantly faster than the $O(N^3)$ complexity of standard $\ell_1$-minimization.
• With high probability $1 - 2N^{-\gamma}$, the reconstruction error $\|\widehat{a} - x\|_{\infty}$ is bounded by $\mu\sigma_1$, ensuring accurate recovery of large coefficients.
• The method is robust to signal model assumptions, as it can be easily adapted to other signal distributions beyond the two-state Gaussian mixture.
• The use of sparse encoding matrices with $L = \Theta(\log N)$ non-zero entries per column enables efficient message passing and maintains near-optimal recovery performance.
• Theoretical analysis confirms that the number of measurements scales as $O(N(1 + 2\eta^{-1}) \cdot \mu^{-2} \cdot [2S + (1-S)(\sigma_0/\sigma_1)^2] \log N)$, balancing sparsity, noise, and accuracy.