[Paper Review] DeepRED: Deep Image Prior Powered by RED

Gary Mataev, Michael Elad+1 more|arXiv (Cornell University)|Mar 25, 2019

Image and Signal Denoising Methods33 references35 citations

TL;DR

DeepRED combines Deep Image Prior with Regularization by Denoising using ADMM to jointly optimize a generative network and a denoiser-based prior, improving unsupervised image restoration results across denoising, super-resolution, and deblurring.

ABSTRACT

Inverse problems in imaging are extensively studied, with a variety of strategies, tools, and theory that have been accumulated over the years. Recently, this field has been immensely influenced by the emergence of deep-learning techniques. One such contribution, which is the focus of this paper, is the Deep Image Prior (DIP) work by Ulyanov, Vedaldi, and Lempitsky (2018). DIP offers a new approach towards the regularization of inverse problems, obtained by forcing the recovered image to be synthesized from a given deep architecture. While DIP has been shown to be quite an effective unsupervised approach, its results still fall short when compared to state-of-the-art alternatives. In this work, we aim to boost DIP by adding an explicit prior, which enriches the overall regularization effect in order to lead to better-recovered images. More specifically, we propose to bring-in the concept of Regularization by Denoising (RED), which leverages existing denoisers for regularizing inverse problems. Our work shows how the two (DIP and RED) can be merged into a highly effective unsupervised recovery process while avoiding the need to differentiate the chosen denoiser, and leading to very effective results, demonstrated for several tested problems.

Motivation & Objective

• Motivate and improve unsupervised inverse problem solving in imaging by strengthening DIP with an explicit regularization prior.
• Leverage Regularization by Denoising (RED) to augment the implicit regularization of DIP.
• Develop an ADMM-based optimization scheme that avoids differentiating the denoiser while training the joint DIP+RED objective.
• Demonstrate enhanced performance of DeepRED over DIP and RED on denoising, SISR, and deblurring.

Proposed method

• Formulate DeepRED objective by combining DIP reconstruction loss with a RED prior: min_{x, Theta} 1/2 ||H T_theta(z) - y||^2_2 + (lambda/2) x^T (x - f(x)) with x = T_theta(z).
• Use Augmented Lagrangian to handle the x = T_theta(z) constraint and derive alternating updates for Theta, x, and the Lagrange multipliers u.
• Update Theta by minimizing the data-consistency term plus a proximity term to keep T_theta(z) close to x - u (Eq. 7).
• Update x via RED-based denoising of T_theta(z) + u, using either a fixed-point iteration (Eq. 11) or a steepest-descent step (Eq. 12).
• Update the dual variable u with the ADMM rule u <- u - x + T_theta(z).
• Employ J=1 RED iterations per ADMM cycle and optionally run the denoiser in parallel with Theta updates to speed up computation.

Experimental results

Research questions

• RQ1 Can RED regularization improve the DIP-based unsupervised image restoration framework?
• RQ2 Does the combination DIP+RED (DeepRED) surpass DIP and RED individually across denoising, single-image super-resolution (SISR), and deblurring?
• RQ3 Is it feasible to optimize the joint DIP+RED objective without differentiating the denoiser, using ADMM?
• RQ4 How does DeepRED perform relative to unsupervised methods and to some supervised approaches in SISR and deblurring?
• RQ5 What are the practical considerations (speed, convergence) when integrating DIP with RED via ADMM?

Key findings

• DeepRED yields improved PSNR over DIP and RED on denoising (e.g., DIP average 30.53 dB vs DeepRED 31.24 dB on the tested images).
• In SISR, DeepRED consistently outperforms both DIP and RED, approaching or matching supervised methods on several datasets and scales (e.g., 4:1 and 8:1 upscaling results shown in Tables 2–4).
• For image deblurring, DeepRED outperforms DIP, RED, and several baselines across multiple blur types and images (e.g., average gains reported in Table 5 for color and Table 6 for grayscale).
• DeepRED demonstrates robust numerical behavior with consistent loss descent and narrowing ADMM constraint gaps, while RED provides regularization that mitigates overfitting in DIP.