[Paper Review] A Systematic Survey of Deep Learning-based Single-Image Super-Resolution

Juncheng Li, Zehua Pei+5 more|arXiv (Cornell University)|Sep 29, 2021

Advanced Image Processing Techniques162 references28 citations

TL;DR

This paper provides a comprehensive survey of deep learning-based single-image super-resolution (SISR), organizing methods by target aims and summarizing datasets, upsampling techniques, losses, and evaluation metrics. It also discusses challenges and future directions in DL-based SISR.

ABSTRACT

Single-image super-resolution (SISR) is an important task in image processing, which aims to enhance the resolution of imaging systems. Recently, SISR has made a huge leap and has achieved promising results with the help of deep learning (DL). In this survey, we give an overview of DL-based SISR methods and group them according to their design targets. Specifically, we first introduce the problem definition, research background, and the significance of SISR. Secondly, we introduce some related works, including benchmark datasets, upsampling methods, optimization objectives, and image quality assessment methods. Thirdly, we provide a detailed investigation of SISR and give some domain-specific applications of it. Fourthly, we present the reconstruction results of some classic SISR methods to intuitively know their performance. Finally, we discuss some issues that still exist in SISR and summarize some new trends and future directions. This is an exhaustive survey of SISR, which can help researchers better understand SISR and inspire more exciting research in this field. An investigation project for SISR is provided at https://github.com/CV-JunchengLi/SISR-Survey.

Motivation & Objective

• Provide a comprehensive overview of DL-based SISR methods organized by specific targets.
• Summarize benchmark datasets, degradation models, and upsampling techniques used in SISR.
• Review optimization objectives and image quality assessment methods in SISR.
• Discuss current challenges, gaps, and future directions to inspire new research.

Proposed method

• Classify DL-based SISR methods into four target-based categories: reconstruction efficiency, reconstruction accuracy, perceptual quality, and further improvement.
• Describe problem setting and degradation models for SISR, including BI, BD, and DN modes.
• Summarize upsampling methods such as pre-upsampling, post-upsampling, and sub-pixel convolution layers.
• Detail loss function types including pixel, content, adversarial, and prior losses, and their roles in guiding training.
• Explain supervised and unsupervised learning strategies and their optimization objectives.
• Discuss assessment methods for reconstruction accuracy, perceptual quality, and reconstruction efficiency.

Figure 1: SISR aims to reconstruct a super-resolution (SR) image from its degraded low-resolution (LR) one.

Experimental results

Research questions

• RQ1 What are the main DL-based SISR methods when categorized by their reconstruction targets?
• RQ2 What benchmark datasets and degradation models are commonly used to train and evaluate SISR models?
• RQ3 What upsampling strategies and loss functions are prevalent, and how do they influence results?
• RQ4 How are DL-based SISR methods evaluated in terms of accuracy, perceptual quality, and efficiency?

Key findings

• The survey covers more than 100 SR methods and highlights a target-based taxonomy for clarity.
• DIV2K is identified as a widely used training dataset, with RealSR representing real-world paired LR/HR data.
• A variety of upsampling methods are discussed, including transposed convolution and sub-pixel convolution, with emphasis on efficiency and accuracy trade-offs.
• Common loss functions include pixel losses (L1/MSE/Charbonnier), content loss, adversarial loss, and prior losses, often used in combination.
• Evaluation covers PSNR/SSIM for accuracy and NIQE/PI/LPIPS/MOS for perceptual quality, noting trade-offs between distortion and perceptual results.
• The paper offers guidance on reconstruction efficiency metrics like model size, execution time, and Mult-Adds to balance performance and practicality.

Figure 2: The content and taxonomy of this survey. In this survey, we divide the DL-based SISR methods into four categories, which are classified according to their specific targets. Among them, the dark gray blocks are the focus methods in this survey.