[Paper Review] DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

Orest Kupyn, Tetiana Martyniuk+2 more|arXiv (Cornell University)|Aug 10, 2019

Advanced Image Processing Techniques50 references112 citations

TL;DR

DeblurGAN-v2 introduces a flexible, backbone-agnostic GAN framework with Feature Pyramid Network and double-scale RaGAN-LS discriminator, achieving state-of-the-art or near state-of-the-art deblurring with much higher efficiency, including real-time capable lightweight backbones.

ABSTRACT

We present a new end-to-end generative adversarial network (GAN) for single image motion deblurring, named DeblurGAN-v2, which considerably boosts state-of-the-art deblurring efficiency, quality, and flexibility. DeblurGAN-v2 is based on a relativistic conditional GAN with a double-scale discriminator. For the first time, we introduce the Feature Pyramid Network into deblurring, as a core building block in the generator of DeblurGAN-v2. It can flexibly work with a wide range of backbones, to navigate the balance between performance and efficiency. The plug-in of sophisticated backbones (e.g., Inception-ResNet-v2) can lead to solid state-of-the-art deblurring. Meanwhile, with light-weight backbones (e.g., MobileNet and its variants), DeblurGAN-v2 reaches 10-100 times faster than the nearest competitors, while maintaining close to state-of-the-art results, implying the option of real-time video deblurring. We demonstrate that DeblurGAN-v2 obtains very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency. Besides, we show the architecture to be effective for general image restoration tasks too. Our codes, models and data are available at: https://github.com/KupynOrest/DeblurGANv2

Motivation & Objective

• Address single-image motion deblurring in a GAN-based end-to-end framework.
• Introduce a Feature Pyramid Network (FPN) as part of the generator to fuse multi-scale features efficiently.
• Enable backbone flexibility to trade off accuracy and speed (e.g., Inception-ResNet-v2 vs MobileNet variants).
• Enhance discriminator design with a double-scale, relativistic RaLSGAN loss for stable training and better perceptual quality.
• Demonstrate efficiency and quality across benchmarks and show applicability to general image restoration challenges.

Proposed method

• Use a conditional GAN framework with a generator incorporating a Feature Pyramid Network to produce sharp images from blurred inputs.
• Adopt a double-scale RaGAN-LS discriminator with global and patch-level (local) branches for multi-scale context.
• Replace DeblurGAN-v1/WGAN-GP with RaGAN-LS loss to stabilize training and improve perceptual quality.
• Make the backbone plug-and-play, enabling backbones like Inception-ResNet-v2 for high quality and MobileNet/MobileNet-DSC for efficiency.
• Train with a hybrid loss L_G = 0.5*L_p + 0.006*L_X + 0.01*L_adv, where L_p is MSE on pixels and L_X is perceptual loss from VGG features.
• Process inputs via data preparation including frame interpolation to synthesize smoother blur for training.

Experimental results

Research questions

• RQ1 Can DeblurGAN-v2 achieve higher or comparable deblurring quality than prior methods while significantly improving inference speed?
• RQ2 How does incorporating an FPN and double-scale RaGAN-LS discriminator affect restoration quality and training stability?
• RQ3 How does the choice of backbone affect the accuracy/efficiency trade-off in DeblurGAN-v2?
• RQ4 Is the framework extendable to general image restoration tasks beyond deblurring?

Key findings

• DeblurGAN-v2 with light backbones (MobileNet, MobileNet-DSC) achieves up to 100x faster inference than prior methods while maintaining competitive SSIM/PSNR.
• DeblurGAN-v2 (Inception-ResNet-v2) attains top quantitative SSIM/PSNR on GoPro, with substantially lower inference time than SRN.
• The MobileNet-DSC variant enables near real-time frame-rate deblurring, achieving 0.04s per image on GoPro-scale data.
• The proposed FPN and double-scale RaGAN-LS discriminator consistently improve PSNR/SSIM over the DeblurGAN baseline in ablations.
• Subjective studies show DeblurGAN-v2 (especially with Inception-ResNet-v2) delivers superior perceptual quality compared to prior methods, while mobile backbones remain visually competitive.
• DeblurGAN-v2 also shows promise for general image restoration tasks beyond deblurring.