[Paper Review] DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency

Zalan Fabian, Berk Tınaz+1 more|PubMed|Mar 25, 2023

Topological and Geometric Data Analysis4 citations

TL;DR

DiracDiffusion proposes a novel diffusion-based framework for inverse problems that models the observation process as a stochastic degradation, enabling data-consistent reconstruction through reverse diffusion without post-processing. It achieves state-of-the-art performance by enabling flexible trade-offs between perceptual quality and distortion metrics via early-stopping, outperforming existing methods on deblurring and inpainting tasks with improved PSNR and LPIPS scores.

ABSTRACT

Diffusion models have established new state of the art in a multitude of computer vision tasks, including image restoration. Diffusion-based inverse problem solvers generate reconstructions of exceptional visual quality from heavily corrupted measurements. However, in what is widely known as the perception-distortion trade-off, the price of perceptually appealing reconstructions is often paid in declined distortion metrics, such as PSNR. Distortion metrics measure faithfulness to the observation, a crucial requirement in inverse problems. In this work, we propose a novel framework for inverse problem solving, namely we assume that the observation comes from a stochastic degradation process that gradually degrades and noises the original clean image. We learn to reverse the degradation process in order to recover the clean image. Our technique maintains consistency with the original measurement throughout the reverse process, and allows for great flexibility in trading off perceptual quality for improved distortion metrics and sampling speedup via early-stopping. We demonstrate the efficiency of our method on different high-resolution datasets and inverse problems, achieving great improvements over other state-of-the-art diffusion-based methods with respect to both perceptual and distortion metrics.

Motivation & Objective

• To address the perception-distortion trade-off in diffusion-based inverse problem solving, where high perceptual quality often comes at the cost of low distortion metrics like PSNR.
• To develop a method that maintains strict data consistency with the original measurement throughout the reverse diffusion process, eliminating the need for post-hoc projections or corrections.
• To enable flexible control over reconstruction quality by allowing early-stopping during sampling, trading perceptual fidelity for improved distortion and faster inference.
• To model the forward degradation process as a stochastic, gradual corruption of the clean image, aligning the diffusion process directly with the physical or algorithmic corruption mechanism.
• To achieve state-of-the-art results across diverse high-resolution inverse problems, including deblurring and inpainting, on CelebA-HQ and ImageNet datasets.

Proposed method

• The method models the forward process as a stochastic degradation process that gradually corrupts the clean image, with the observation being a noisy version of the original signal.
• It learns to reverse this specific degradation process using a diffusion model trained on the same stochastic corruption dynamics, ensuring inherent data consistency.
• The reverse process maintains consistency with the original measurement by construction, without requiring additional projection steps or iterative corrections.
• The framework enables flexible reconstruction quality by allowing early-stopping at any step in the reverse sampling trajectory, trading off perceptual quality for improved distortion and speed.
• The method leverages the gradual addition of details during sampling, where later steps refine high-frequency components, enabling control over the final output quality.
• It is trained end-to-end on the degradation process, with the forward SDE explicitly encoding the measurement model, allowing direct application to various inverse problems.

Experimental results

Research questions

• RQ1 Can a diffusion model trained on a problem-specific stochastic degradation process achieve better data consistency and distortion metrics than standard diffusion models?
• RQ2 How does early-stopping in the reverse sampling process affect the trade-off between perceptual quality and distortion in inverse problem reconstruction?
• RQ3 Can a unified reverse sampling trajectory produce multiple reconstructions with varying perceptual and distortion characteristics without re-running the process?
• RQ4 Does modeling the forward degradation as a stochastic process that matches the actual corruption mechanism lead to improved reconstruction fidelity compared to generic Gaussian noise addition?
• RQ5 To what extent can the proposed method outperform existing diffusion-based inverse solvers in terms of PSNR, SSIM, and LPIPS on high-resolution datasets?

Key findings

• DiracDiffusion achieves state-of-the-art performance on both deblurring and inpainting tasks across CelebA-HQ and ImageNet datasets, surpassing existing diffusion-based methods in both perceptual and distortion metrics.
• The method achieves significantly improved PSNR scores—up to 1.5 dB higher than baseline methods—while maintaining or improving perceptual quality as measured by LPIPS.
• By enabling early-stopping, the framework allows users to obtain reconstructions with higher distortion fidelity (e.g., higher PSNR) at the cost of reduced perceptual quality, with minimal computational overhead.
• The uncertainty maps across 10 samples show low variance in reconstructions, indicating stable and consistent sampling behavior, especially in early-stopping regimes.
• The method outperforms strong baselines such as DPS, DDRM, PnP-ADMM, and ADMM-TV in both quantitative metrics and visual quality, particularly in high-frequency detail recovery.
• The framework demonstrates robustness across diverse inverse problems, including Gaussian deblurring and inpainting, with consistent improvements on high-resolution data.