[Paper Review] Designing a Practical Degradation Model for Deep Blind Image Super-Resolution

Kai Zhang, Jingyun Liang+2 more|arXiv (Cornell University)|Mar 25, 2021

Infrared Target Detection MethodologiesEngineering41 citations

TL;DR

The paper introduces a large, realistic degradation model with randomly shuffled blur, downsampling, and noise to train a deep blind SISR model, improving performance on real-world degradations. It trains BSRNet/BSRGAN using synthetic paired data generated from high-res images.

ABSTRACT

It is widely acknowledged that single image super-resolution (SISR) methods would not perform well if the assumed degradation model deviates from those in real images. Although several degradation models take additional factors into consideration, such as blur, they are still not effective enough to cover the diverse degradations of real images. To address this issue, this paper proposes to design a more complex but practical degradation model that consists of randomly shuffled blur, downsampling and noise degradations. Specifically, the blur is approximated by two convolutions with isotropic and anisotropic Gaussian kernels; the downsampling is randomly chosen from nearest, bilinear and bicubic interpolations; the noise is synthesized by adding Gaussian noise with different noise levels, adopting JPEG compression with different quality factors, and generating processed camera sensor noise via reverse-forward camera image signal processing (ISP) pipeline model and RAW image noise model. To verify the effectiveness of the new degradation model, we have trained a deep blind ESRGAN super-resolver and then applied it to super-resolve both synthetic and real images with diverse degradations. The experimental results demonstrate that the new degradation model can help to significantly improve the practicability of deep super-resolvers, thus providing a powerful alternative solution for real SISR applications.

Motivation & Objective

• Motivate and design a degradation model that better captures real image degradations beyond bicubic or simple traditional models.
• Enable generation of unlimited paired LR-HR data without alignment issues for supervised blind SISR training.
• Train and evaluate deep blind SISR networks (BSRNet/BSRGAN) using the proposed degradation model.
• Demonstrate improved practicability of blind SR on synthetic and real degraded images.

Proposed method

• Propose a degradation model with three key factors (blur, downsampling, noise) expanded via isotropic and anisotropic Gaussian blurs, multiple downsampling operators, and diverse noise types (Gaussian, JPEG compression, processed camera sensor noise).
• Introduce a degradation shuffle strategy that randomly orders the degradation steps to expand the degradation space.
• Model HR blur and LR blur via two Gaussian convolutions; apply four downsampling options (nearest, bilinear, bicubic, down-up); synthesize noise with AWGN, JPEG noise, and processed RAW/ISP-based noise.
• Use random sampling of kernel sizes, blur widths, rotation angles, and downsampling scales to generate a wide variety of LR images.
• Apply two JPEG compressions (with quality in [30,95]) and a final JPEG step to mimic real-world artifacts.
• Leverage a reverse-forward ISP pipeline to synthesize camera sensor noise from RAW data, using multiple real cameras to form an ISP pool.

Experimental results

Research questions

• RQ1 Can a functionally richer, randomly shuffled degradation model cover the real degradation space better than traditional models?
• RQ2 Does training blind SR models on data generated from the proposed degradation model improve performance on real degraded images?
• RQ3 How do blind SR networks trained with the new degradation model compare to existing methods on synthetic and real datasets?
• RQ4 What are the qualitative and quantitative benefits of using BSRNet/BSRGAN with the proposed degradation model?

Key findings

• The proposed degradation model expands the degradation space beyond bicubic/traditional shapes by combining two blur convolutions, multiple downsampling methods, and varied noise types.
• Random degradation shuffle enables the model to cover diverse real-world degradations and subsumes several existing degradation schemes as special cases.
• BSRNet achieves the best overall PSNR on synthetic DIV2K4D data, while BSRGAN achieves the best perceptual (LPIPS) results across tested degradations.
• On RealSRSet real images, BSRNet provides stronger non-reference IQA performance and visually cleaner results than competing methods.
• The study highlights the importance of accurate degradation modeling for practical SISR applications and shows that large-capacity networks can generalize across diverse degradations.