[Paper Review] Image Super-Resolution Using VDSR-ResNeXt and SRCGAN

Saifuddin Hitawala, Yao Li+2 more|arXiv (Cornell University)|Oct 10, 2018

Advanced Image Processing Techniques12 references5 citations

TL;DR

This paper proposes two deep learning-based image super-resolution methods: VDSR-ResNeXt, a residual, multi-branch convolutional network combining VDSR and ResNeXt architectures for improved parameter efficiency and performance, and SRCGAN, a conditional GAN that uses class labels to generate high-resolution images. The VDSR-ResNeXt model reduces parameters by 30% compared to standard VDSR while maintaining competitive PSNR, and SRCGAN significantly improves image clarity and classifier accuracy over vanilla GANs on MNIST, achieving 81.26% digit recognition accuracy on generated images.

ABSTRACT

Over the past decade, many Super Resolution techniques have been developed using deep learning. Among those, generative adversarial networks (GAN) and very deep convolutional networks (VDSR) have shown promising results in terms of HR image quality and computational speed. In this paper, we propose two approaches based on these two algorithms: VDSR-ResNeXt, which is a deep multi-branch convolutional network inspired by VDSR and ResNeXt; and SRCGAN, which is a conditional GAN that explicitly passes class labels as input to the GAN. The two methods were implemented on common SR benchmark datasets for both quantitative and qualitative assessment.

Motivation & Objective

• To improve image super-resolution performance by combining the residual learning of VDSR with the multi-branch, grouped convolution design of ResNeXt.
• To explore the effectiveness of conditional GANs in super-resolution by incorporating class labels as input to the generator.
• To evaluate both models on benchmark datasets using quantitative metrics (PSNR, SSIM) and qualitative visual assessment.
• To investigate whether conditional supervision enhances the realism and recognizability of reconstructed high-resolution images.
• To analyze the impact of architectural choices such as grouped convolutions, residual blocks, and training hyperparameters on model performance.

Proposed method

• VDSR-ResNeXt uses a multi-branch residual block design inspired by ResNeXt, where each block consists of 32 grouped convolutional branches, each processing a subset of input channels before being summed.
• The model employs a split-transform-merge strategy using grouped convolutions to reduce parameter count while maintaining representational capacity.
• The loss function is mean squared error: $ \frac{1}{2}||\mathbf{y} - f(\mathbf{x})||^2 $, where $ f(\mathbf{x}) $ is the predicted HR image and $ \mathbf{y} $ is the ground truth.
• SRCGAN is a conditional GAN that takes both a low-resolution image and a class label (e.g., digit identity) as input to the generator, improving reconstruction fidelity.
• The discriminator in SRCGAN is trained to distinguish between real HR images and generated HR images conditioned on the same label.
• Training uses Adam optimizer with a batch size of 128 and a learning rate of 0.001 for both generator and discriminator, with 100 epochs for SRCGAN and 70 for the classifier.

Experimental results

Research questions

• RQ1 Can the multi-branch, grouped convolution architecture of ResNeXt improve the performance and parameter efficiency of the VDSR framework in image super-resolution?
• RQ2 Does incorporating class labels as conditional input in a GAN framework lead to sharper and more semantically accurate high-resolution image reconstructions compared to unconditional GANs?
• RQ3 How do the proposed models compare to the original VDSR and vanilla GANs in terms of PSNR, SSIM, and visual quality across different scale factors?
• RQ4 To what extent does conditional supervision improve the recognizability of generated images, as measured by downstream classification accuracy?
• RQ5 What are the effects of training duration, learning rate scheduling, and data distribution on the performance of the proposed models?

Key findings

• VDSR-ResNeXt with 18 layers and 256 filters achieved a PSNR of 32.15 dB on the Set5 dataset at scale factor 2, comparable to the original VDSR, while reducing parameters by approximately 30%.
• The VDSR-ResNeXt model reduced the number of parameters from ~885K to ~313K for a 64→256→64 block configuration by using 32 grouped convolution branches.
• SRCGAN achieved a digit classification accuracy of 81.26% on generated images from the MNIST dataset, significantly higher than the 68.11% accuracy for images from a vanilla GAN.
• Qualitative results show that SRCGAN generates clearer, more recognizable digits compared to the blurry outputs of the vanilla GAN, especially at scale factor 4.
• The discriminator and generator losses in SRCGAN converged stably over 100 training epochs, indicating effective adversarial training with conditional supervision.
• The authors observed that the model's performance lags behind VDSR as the scale factor increases, suggesting a need for more data at higher magnification factors.