[Paper Review] Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration

Mauricio Delbracio, Peyman Milanfar|arXiv (Cornell University)|Mar 20, 2023

Photoacoustic and Ultrasonic ImagingEngineering30 citations

TL;DR

InDI iteratively restores degraded images by small steps, avoiding single-step regression to the mean and improving perceptual quality across motion/deblurring, defocus, super-resolution, and compression artifact tasks. It learns an iterative restoration function conditioned on time t and does not require a known degradation model.

ABSTRACT

Inversion by Direct Iteration (InDI) is a new formulation for supervised image restoration that avoids the so-called "regression to the mean" effect and produces more realistic and detailed images than existing regression-based methods. It does this by gradually improving image quality in small steps, similar to generative denoising diffusion models. Image restoration is an ill-posed problem where multiple high-quality images are plausible reconstructions of a given low-quality input. Therefore, the outcome of a single step regression model is typically an aggregate of all possible explanations, therefore lacking details and realism. The main advantage of InDI is that it does not try to predict the clean target image in a single step but instead gradually improves the image in small steps, resulting in better perceptual quality. While generative denoising diffusion models also work in small steps, our formulation is distinct in that it does not require knowledge of any analytic form of the degradation process. Instead, we directly learn an iterative restoration process from low-quality and high-quality paired examples. InDI can be applied to virtually any image degradation, given paired training data. In conditional denoising diffusion image restoration the denoising network generates the restored image by repeatedly denoising an initial image of pure noise, conditioned on the degraded input. Contrary to conditional denoising formulations, InDI directly proceeds by iteratively restoring the input low-quality image, producing high-quality results on a variety of image restoration tasks, including motion and out-of-focus deblurring, super-resolution, compression artifact removal, and denoising.

Motivation & Objective

• Address the regression-to-the-mean limitation in one-shot image restoration
• Propose an iterative restoration framework that gradually improves image quality
• Avoid reliance on an explicit degradation model by using paired training data
• Show applicability across multiple restoration tasks (motion/deblurring, SR, compression artifacts)
• Link the method to diffusion-style ideas while remaining model- and degradation-agnostic

Proposed method

• Define a forward degradation x_t = (1-t)x + t y with t in [0,1] to create intermediate states between high-quality x and low-quality y
• Train a single conditional regressor F_theta(x_t, t) to predict x from x_t at each time step
• Iteratively update x_{t-δ} = (δ/t)F_theta(x_t, t) + (1−δ/t)x_t with δ controlling step size (often δ=1/N)
• Optional stochastic perturbations add noise to inputs to ensure well-defined posterior expectations and improve perceptual quality
• In the continuous limit, derive a residual-flow ODE dx_t/dt = (x_t − F_theta(x_t, t))/t and relate to score-based DSM in Gaussian noise cases
• Provide an algorithm for inference (Algorithm 1) that starts from y (or y with small noise) and iterates to x_0

Experimental results

Research questions

• RQ1 Can iterative, small-step restoration outperform single-step regression in preserving texture and realism?
• RQ2 Is it possible to restore diverse, high-quality images without a known degradation model by learning from paired data?
• RQ3 How does the number of inference steps affect the perception-distortion trade-off across tasks?
• RQ4 How does InDI relate to diffusion-based approaches, and can it reproduce DSM-like behavior under Gaussian degradation?
• RQ5 Does adding controlled noise to inputs during restoration improve perceptual outcomes?

Key findings

• Outperforms single-step regression in perceptual quality across tasks, approaching diffusion-model results while remaining simpler and degradation-agnostic
• On GoPro motion deblurring, the method achieves state-of-the-art perceptual metrics with competitive PSNR/SSIM
• Increasing the number of steps improves perceptual metrics at the expense of PSNR, illustrating the perception–distortion trade-off
• For 4× super-resolution on div2k, achieves competitive PSNR and better perceptual scores (lower LPIPS) than several regression-based methods and closely matches diffusion-based approaches
• The method generalizes to motion/deblurring, defocus deblurring, JPEG artifact removal, and super-resolution, with analyses showing the benefit of stepwise restoration and occasional noise perturbation
• In Gaussian degradation settings, the method’s ODE aligns with known score-matching diffusion dynamics, illustrating theoretical coherence