[Paper Review] Diffusion Posterior Sampling for General Noisy Inverse Problems

Hyungjin Chung, Jeongsol Kim+3 more|arXiv (Cornell University)|Sep 29, 2022

Advanced X-ray Imaging Techniques151 citations

TL;DR

Diffusion Posterior Sampling (DPS) generalizes diffusion-model based solvers to handle general noisy inverse problems, including nonlinear forward models and measurement noise (Gaussian and Poisson), without relying on SVD, achieving high-quality reconstructions across diverse tasks.

ABSTRACT

Diffusion models have been recently studied as powerful generative inverse problem solvers, owing to their high quality reconstructions and the ease of combining existing iterative solvers. However, most works focus on solving simple linear inverse problems in noiseless settings, which significantly under-represents the complexity of real-world problems. In this work, we extend diffusion solvers to efficiently handle general noisy (non)linear inverse problems via approximation of the posterior sampling. Interestingly, the resulting posterior sampling scheme is a blended version of diffusion sampling with the manifold constrained gradient without a strict measurement consistency projection step, yielding a more desirable generative path in noisy settings compared to the previous studies. Our method demonstrates that diffusion models can incorporate various measurement noise statistics such as Gaussian and Poisson, and also efficiently handle noisy nonlinear inverse problems such as Fourier phase retrieval and non-uniform deblurring. Code available at https://github.com/DPS2022/diffusion-posterior-sampling

Motivation & Objective

• Motivate solving general noisy (non)linear inverse problems with diffusion priors.
• Develop a tractable posterior sampling method that incorporates measurement likelihood without explicit time-dependent likelihoods.
• Enable handling of Gaussian and Poisson noise within a unified DPS framework.
• Avoid costly SVD by operating fully in the image domain for efficient implementation.
• Demonstrate DPS on diverse problems including inpainting, super-resolution, deblurring, and nonlinear phase retrieval.

Proposed method

• Use VP-SDE/DDPMs to model data priors via a denoising score network s_theta* trained by denoising score matching.
• Derive a tractable posterior sampling scheme by approximating p(y|x_t) with p(y|x_hat_0), where x_hat_0 is the posterior mean E[x_0|x_t] computed via Tweedie-style relations: x_hat_0 ≈ (1/√(ᾱ(t))) (x_t + (1−ᾱ(t)) s_theta*(x_t, t)).
• Incorporate the likelihood gradient ∇_{x_t} log p_t(x_t|y) as s_theta*(x_t, t) − ρ ∇_{x_t} || y − A(x_hat_0) ||^2 (Gaussian) or ∇_{x_t} log p_t(x_t|y) ≈ s_theta*(x_t, t) − ρ ∇_{x_t} || y − A(x_hat_0) ||^2_{Λ} (Poisson), where ρ = 1/σ^2 and Λ encodes Poisson weights.
• Operate entirely in the image domain to avoid SVD and allow nonlinear forward models A(·).
• Provide discrete algorithms (DPS-Gaussian and DPS-Poisson) with step-size choices that stabilize sampling under noisy measurements.

Experimental results

Research questions

• RQ1 Can diffusion-based priors be extended to general noisy inverse problems with nonlinear forward models?
• RQ2 How can one approximate the intractable p(y|x_t) to enable posterior sampling without relying on measurement subspace projections?
• RQ3 Do Gaussian and Poisson measurement noises, including nonlinear forward models, admit tractable DPS formulations?
• RQ4 Does DPS yield superior reconstructions over projection-based or SVD-dependent methods under noisy conditions?
• RQ5 Can the framework handle tasks beyond linear inverse problems, such as Fourier phase retrieval and non-uniform deblurring?

Key findings

• DPS achieves high-quality reconstructions across noisy, linear, and nonlinear inverse problems.
• The method outperforms several baselines (e.g., DDRM, MCG, PnP-ADMM, Score-SDE) on tasks like inpainting, super-resolution, and various deblurring settings.
• The framework accommodates Gaussian and Poisson noise via analytically tractable likelihood gradients.
• DPS operates in the image domain, avoiding SVD and enabling efficient handling of complex forward models.
• For nonlinear phase retrieval and nonlinear deblurring, DPS delivers competitive or superior perceptual quality (FID/LPIPS) compared to existing diffusion-based or prior-driven approaches.