[Paper Review] Direct Diffusion Bridge using Data Consistency for Inverse Problems

Hyungjin Chung, Jeongsol Kim+1 more|arXiv (Cornell University)|May 31, 2023

Numerical methods in inverse problems4 citations

TL;DR

This paper proposes Data Consistent Direct Diffusion Bridge (CDDB), a train-free inference method that enforces data consistency in diffusion-based inverse problem solvers by injecting gradient steps during reverse sampling. By unifying existing Direct Diffusion Bridge (DDB) methods and introducing a consistency-improving mechanism without fine-tuning, CDDB achieves state-of-the-art performance across perception and distortion metrics, pushing the Pareto frontier toward optimal reconstruction quality on diverse inverse problems including image super-resolution and JPEG restoration.

ABSTRACT

Diffusion model-based inverse problem solvers have shown impressive performance, but are limited in speed, mostly as they require reverse diffusion sampling starting from noise. Several recent works have tried to alleviate this problem by building a diffusion process, directly bridging the clean and the corrupted for specific inverse problems. In this paper, we first unify these existing works under the name Direct Diffusion Bridges (DDB), showing that while motivated by different theories, the resulting algorithms only differ in the choice of parameters. Then, we highlight a critical limitation of the current DDB framework, namely that it does not ensure data consistency. To address this problem, we propose a modified inference procedure that imposes data consistency without the need for fine-tuning. We term the resulting method data Consistent DDB (CDDB), which outperforms its inconsistent counterpart in terms of both perception and distortion metrics, thereby effectively pushing the Pareto-frontier toward the optimum. Our proposed method achieves state-of-the-art results on both evaluation criteria, showcasing its superiority over existing methods. Code is available at https://github.com/HJ-harry/CDDB

Motivation & Objective

• To identify and address the critical lack of data consistency in existing Direct Diffusion Bridge (DDB) methods for inverse problems.
• To unify disparate DDB approaches—rooted in Schrödinger bridge, continual degradation, and OU-SDE—under a single theoretical framework.
• To develop a modified inference procedure that enforces data consistency without retraining, thereby improving reconstruction fidelity and perceptual quality.
• To demonstrate that the proposed method outperforms existing DDB and diffusion-based solvers across multiple inverse imaging tasks.

Proposed method

• Proposes a unified framework for Direct Diffusion Bridges (DDB), where the reverse diffusion process directly maps from corrupted measurements to clean images via a time-conditional denoiser.
• Introduces a train-free inference procedure that enforces data consistency by applying gradient steps toward the measurement during reverse sampling, without modifying the pre-trained model.
• Derives CDDB as a generalization of existing diffusion sampling methods like DDS and DPS, extending their principles to the DDB setting.
• Employs a consistent sampling strategy that minimizes the distance between the current sample and the measurement, using a Gaussian likelihood-based regularization.
• Proposes CDDB-deep as an extension that uses deeper gradients, enabling application to nonlinear inverse problems such as JPEG restoration where standard gradients are intractable.
• Uses a convex combination of clean and corrupted data to define the forward diffusion process in DDB, ensuring smooth degradation from clean to measured data.

Experimental results

Research questions

• RQ1 How can data consistency be enforced in Direct Diffusion Bridge (DDB) methods without requiring model fine-tuning or retraining?
• RQ2 To what extent does enforcing data consistency improve the trade-off between perceptual quality and reconstruction distortion in diffusion-based inverse problems?
• RQ3 Can the proposed method generalize across diverse inverse problems, including linear and nonlinear degradation models?
• RQ4 How does the performance of CDDB compare to existing state-of-the-art diffusion-based solvers in terms of both FID and PSNR metrics?
• RQ5 Can the CDDB framework be extended to non-Gaussian or nonlinear inverse problems such as JPEG restoration, where gradient computation is not straightforward?

Key findings

• CDDB achieves state-of-the-art performance on both FID and PSNR metrics across multiple inverse problems, including 4x super-resolution and JPEG restoration.
• On the SR4x-bicubic task, CDDB improves PSNR by up to 0.5 dB and reduces FID by over 10% compared to the baseline I2SB (a DDB method), demonstrating superior trade-off between perception and distortion.
• CDDB effectively removes noise from reconstructions, unlike I2SB, which tends to propagate noise—especially in background regions—highlighting its robustness.
• The method shows strong noise robustness even when trained on noise-free data, due to the implicit Gaussian likelihood model embedded in the consistency step.
• CDDB-deep enables effective application to nonlinear inverse problems such as JPEG restoration, where standard gradient-based methods fail, by leveraging deeper gradient approximations.
• The Pareto frontier of reconstruction quality is significantly shifted toward the optimal region, with CDDB achieving higher perceptual quality at lower distortion than all baselines.