[论文解读] Denoising Diffusion Restoration Models

Bahjat Kawar, Michael Elad+2 more|arXiv (Cornell University)|Jan 27, 2022

Image and Signal Denoising Methods被引用 240

一句话总结

DDR M 是一种基于扩散的无监督求解器，用于通用线性逆问题，利用预训练扩散模型和基于 SVD 的问题条件，在迭代次数较少的情况下实现高质量图像恢复。

ABSTRACT

Many interesting tasks in image restoration can be cast as linear inverse problems. A recent family of approaches for solving these problems uses stochastic algorithms that sample from the posterior distribution of natural images given the measurements. However, efficient solutions often require problem-specific supervised training to model the posterior, whereas unsupervised methods that are not problem-specific typically rely on inefficient iterative methods. This work addresses these issues by introducing Denoising Diffusion Restoration Models (DDRM), an efficient, unsupervised posterior sampling method. Motivated by variational inference, DDRM takes advantage of a pre-trained denoising diffusion generative model for solving any linear inverse problem. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization under various amounts of measurement noise. DDRM outperforms the current leading unsupervised methods on the diverse ImageNet dataset in reconstruction quality, perceptual quality, and runtime, being 5x faster than the nearest competitor. DDRM also generalizes well for natural images out of the distribution of the observed ImageNet training set.

研究动机与目标

• 在不重新训练的情况下，为多样化的降解模型提供无监督的恢复动机。
• 利用预训练的扩散模型作为先验，从逆问题的后验中进行采样。
• 开发一个变分推断框架，使扩散去噪与线性逆问题约束对齐。
• 提出通过 SVD 条件化的光谱空间扩散，以处理各种线性算子。
• 在跨任务和跨数据集上展示效率与鲁棒性，包括对分布外图像的泛化。

提出的方法

• 为以观测值为条件的线性逆问题制定后验采样目标。
• 通过 SVD 在降解算子的光谱空间中进行扩散，以分离观测与缺失分量。
• 将 p_theta 和 q_t 定义为高斯条件分布，以在 DDRM 下匹配 DDPM/DDIM 式目标，从而实现无条件扩散模型的重用。
• 在合理假设下证明 DDRM 目标与 DDPM/DDIM 等价，从而允许用单一扩散模型处理多种问题。
• 引入效率提升：使用子时间步的加速采样、内存高效的 SVD，以及与预训练扩散模型的兼容性。
• 给出用于解决恢复任务的实用算法步骤（更新规则），在 20-100 NFEs 内完成。

实验结果

研究问题

• RQ1 Can DDRM solve a wide range of linear inverse problems unsupervised, without re-training for each degradation model?
• RQ2 How can diffusion models be leveraged in the spectral domain of the degradation operator to faithfully reconstruct missing data while respecting measurements?
• RQ3 What are the performance, speed, and memory characteristics of DDRM compared to other unsupervised priors on diverse datasets?
• RQ4 Is a single diffusion model capable of handling multiple restoration tasks (super-resolution, deblurring, inpainting, colorization) including noisy measurements?

主要发现

Method	4x Super-Resolution PSNR	4x Super-Resolution SSIM	4x Super-Resolution KID	4x Super-Resolution NFEs	Deblurring PSNR	Deblurring SSIM	Deblurring KID	Deblurring NFEs
Baseline	25.65	0.71	44.90	0	19.26	0.48	38.00	0
DGP	23.06	0.56	21.22	1500	22.70	0.52	27.60	1500
RED	26.08	0.73	53.55	100	26.16	0.76	21.21	500
SNIPS	17.58	0.22	35.17	1000	34.32	0.87	0.49	1000
DDRM	26.55	0.72	7.22	20	35.64	0.95	0.71	20
DDRM-CC	26.55	0.74	6.56	20	35.65	0.96	0.70	20

• DDRM outperforms leading unsupervised priors on ImageNet across multiple tasks and often with far fewer NFEs (e.g., 20 vs 1500).
• DDRMs can achieve competitive PSNR/SSIM and much lower KID than baselines in noiseless and noisy settings.
• DDRM demonstrates strong qualitative results, producing diverse valid restorations and generalizing to out-of-distribution natural images.
• Using SVD, the method handles various degradation operators efficiently with memory usage optimized to O(n) in many cases.
• Theoretical result: with appropriate scheduling, the DDRM objective aligns with DDPM/DDIM objectives, enabling reuse of unconditional diffusion models for conditioned restoration.
• DDRMs maintain performance across different datasets and degradation models, demonstrating flexibility without task-specific retraining.