[論文レビュー] Generative Diffusion Prior for Unified Image Restoration and Enhancement

Ben Fei, Zhaoyang Lyu+6 more|arXiv (Cornell University)|Apr 3, 2023

Advanced Image Processing Techniques被引用数 10

ひとこと要約

GDP は、統一的で教師なしの prior として、条件付きガイダンスと degradation model optimization を介して、線形・非線形・ブラインド画像復元と強化を実行するための pre-trained denoising diffusion probabilistic model を使用します。

ABSTRACT

Existing image restoration methods mostly leverage the posterior distribution of natural images. However, they often assume known degradation and also require supervised training, which restricts their adaptation to complex real applications. In this work, we propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems. Specifically, GDP systematically explores a protocol of conditional guidance, which is verified more practical than the commonly used guidance way. Furthermore, GDP is strength at optimizing the parameters of degradation model during the denoising process, achieving blind image restoration. Besides, we devise hierarchical guidance and patch-based methods, enabling the GDP to generate images of arbitrary resolutions. Experimentally, we demonstrate GDP's versatility on several image datasets for linear problems, such as super-resolution, deblurring, inpainting, and colorization, as well as non-linear and blind issues, such as low-light enhancement and HDR image recovery. GDP outperforms the current leading unsupervised methods on the diverse benchmarks in reconstruction quality and perceptual quality. Moreover, GDP also generalizes well for natural images or synthesized images with arbitrary sizes from various tasks out of the distribution of the ImageNet training set.

研究の動機と目的

• Motivate a unified, unsupervised prior for diverse image restoration and enhancement tasks beyond known degradations.
• Leverage a pre-trained DDPM as a prior to solve linear, non-linear, and blind degradation problems in a single framework.
• Enable arbitrary-size image restoration through hierarchical guidance and patch-based strategies.
• Allow simultaneous estimation of unknown degradation models during the denoising process to achieve blind restoration.
• Demonstrate versatility across datasets (ImageNet, LSUN, CelebA) and tasks (SR, deblurring, inpainting, colorization, low-light enhancement, HDR) with competitive quality metrics.

提案手法

• GDP は、pre-trained unconditional DDPM を prior として使用し、復元を実行するために degraded observations に対して its reverse process を条件付ける。
• p_theta(x_{t-1}|x_t,y) を、unconditional な遷移を data-consistency term によって導くことで近似する probabilistic なフレームワークを採用する（p(y|x_t) から導かれる）。
• L 的な reconstruction term と、任意で quality-enhancement term Q を導入し、劣化パラメータの勾配ベース更新を介して劣化認識サンプリングを誘導する。
• 2 つの guidance variant を提示する：GDP-x_t（x_t に対する guidance）と GDP-tilde{x}_0（予測されたクリーン画像に対する guidance）、品質向上のための分散除去について議論する。
• 未知の（ブラインド）劣化へ GDP を拡張するために、Algorithm 2 のように reverse diffusion の各ステップで劣化パラメータ φ を同時最適化する。
• patch-based な生成スキームと階層的 guidance 戦略、HDR を含むマルチ画像 guidance を含む、任意解像度出力を可能にする。

実験結果

リサーチクエスチョン

• RQ1 1. 1つの事前学習済み拡散モデルは、再学習なしで複数の復元・強化タスクに対する汎用的な prior となり得るか？
• RQ2 2. 条件付きガイダンスと効果的な degradation パラメータの共同最適化は、拡散フレームワーク内でブラインドで非線形な画像復元を可能にするか？
• RQ3 3. 階層的およびパッチベースの戦略は、任意の画像サイズで高品質な復元を可能にするか？
• RQ4 4. GDP は、線形・非線形・ブラインドタスクを、標準ベンチマーク上の非教師なしベースラインと比べてどう機能するか？
• RQ5 5. GDP は、分布外入力に対して頑健で、HDR などのマルチ画像ガイダンスを扱えるか？

主な発見

• GDPOutperforms leading unsupervised methods on several linear inverse problems in terms of reconstruction quality and perceptual metrics.
• GDP can handle non-linear and blind restoration tasks by iteratively estimating and updating degradation parameters during diffusion denoising.
• Hierarchical guidance and patch-based generation enable restoration of arbitrary-resolution images with competitive fidelity.
• GDP demonstrates strong generalization to out-of-distribution natural or synthesized images and maintains performance across diverse datasets (ImageNet, LSUN, CelebA).
• The approach yields high-fidelity and faithful reconstructions across SR, deblurring, inpainting, colorization, low-light enhancement, and HDR recovery, often surpassing zero-shot baselines.