[Paper Review] Instance-Optimal Compressed Sensing via Posterior Sampling

Ajil Jalal, Sushrut Karmalkar+2 more|arXiv (Cornell University)|Jun 21, 2021

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper proposes posterior sampling via Langevin dynamics as an instance-optimal compressed sensing method for signals from any known prior distribution, achieving near-optimal recovery with Gaussian measurements. It proves that posterior sampling matches the theoretical lower bound on measurement complexity and remains robust under distributional model mismatch when the estimated prior is close in Wasserstein distance.

ABSTRACT

We characterize the measurement complexity of compressed sensing of signals drawn from a known prior distribution, even when the support of the prior is the entire space (rather than, say, sparse vectors). We show for Gaussian measurements and \emph{any} prior distribution on the signal, that the posterior sampling estimator achieves near-optimal recovery guarantees. Moreover, this result is robust to model mismatch, as long as the distribution estimate (e.g., from an invertible generative model) is close to the true distribution in Wasserstein distance. We implement the posterior sampling estimator for deep generative priors using Langevin dynamics, and empirically find that it produces accurate estimates with more diversity than MAP.

Motivation & Objective

• To characterize the measurement complexity required for compressed sensing when signals are drawn from an arbitrary known prior distribution, not just sparse or structured sets.
• To establish that posterior sampling achieves instance-optimal recovery guarantees across all distributions, including those with full support over R^n.
• To analyze robustness to prior model mismatch, showing that accurate recovery is possible if the estimated prior is close to the true prior in Wasserstein distance.
• To provide a practical implementation of posterior sampling using Langevin dynamics for deep generative priors, demonstrating improved diversity and accuracy over MAP.

Proposed method

• Proposes posterior sampling as a recovery algorithm that draws samples from the posterior distribution p(x|y,A), where y is the measurement vector and A is the measurement matrix.
• Uses a novel concept of approximate covering numbers to characterize the measurement complexity required for recovery under any prior distribution.
• Establishes an upper bound on measurement complexity for posterior sampling in terms of the approximate covering number of the prior distribution.
• Proves that the measurement complexity of posterior sampling is near-optimal by deriving a matching instance-specific lower bound.
• Employs Langevin dynamics to implement posterior sampling for deep generative models, using a schedule of decreasing noise and gradient steps.
• Demonstrates robustness by showing that if the estimated prior P is close to the true prior R in Wasserstein distance, then posterior sampling with P still yields accurate recovery.

Experimental results

Research questions

• RQ1 Can posterior sampling achieve instance-optimal compressed sensing performance for any signal distribution, even when the support is the entire R^n?
• RQ2 What is the minimal number of Gaussian measurements required to achieve near-optimal recovery for a given prior distribution?
• RQ3 How robust is posterior sampling to errors in the prior model, particularly when the estimated prior is close to the true prior in Wasserstein distance?
• RQ4 Can posterior sampling via Langevin dynamics produce more diverse and accurate reconstructions than MAP in image reconstruction tasks?
• RQ5 How does the performance of posterior sampling compare to MAP and deep decoder baselines in compressed sensing and inpainting settings?

Key findings

• Posterior sampling achieves near-optimal recovery guarantees for any prior distribution, with measurement complexity bounded by O(m log(1 + mr²‖A‖∞²/σ²) + log(1/δ)) for high-probability recovery.
• The method is robust to model mismatch: if the estimated prior P is within O(σδ¹ᐟᵖ) Wasserstein distance of the true prior R, then posterior sampling with P still yields accurate recovery.
• Empirical results show that Langevin-based posterior sampling produces diverse, high-quality reconstructions in compressed sensing and inpainting tasks, outperforming MAP in visual quality and diversity.
• In experiments on FFHQ and CelebA-HQ, posterior sampling with NCSNv2 and Glow models achieved better reconstruction quality than MAP and Deep-Decoder baselines.
• Theoretical analysis confirms that posterior sampling matches the instance-optimal lower bound on measurement complexity, making it information-theoretically optimal for any given prior.
• The use of Langevin dynamics enables practical posterior sampling for complex deep generative priors, overcoming the limitations of MAP-based approaches which often produce washed-out, low-diversity outputs.