[Paper Review] Single Image Super-Resolution via Residual Neuron Attention Networks

Wenjie Ai, Xiaoguang Tu+2 more|arXiv (Cornell University)|May 21, 2020

Advanced Image Processing Techniques26 references4 citations

TL;DR

This paper proposes Residual Neuron Attention Networks (RNAN) for single image super-resolution, leveraging Global Context-enhanced Residual Groups (GCRGs) with a novel Residual Neuron Attention (RNA) mechanism to model neuron-wise feature dependencies and global context. RNAN achieves state-of-the-art performance with fewer parameters, outperforming methods like EDSR and RDN on benchmark datasets while maintaining high visual fidelity and quantitative metrics.

ABSTRACT

Deep Convolutional Neural Networks (DCNNs) have achieved impressive performance in Single Image Super-Resolution (SISR). To further improve the performance, existing CNN-based methods generally focus on designing deeper architecture of the network. However, we argue blindly increasing network's depth is not the most sensible way. In this paper, we propose a novel end-to-end Residual Neuron Attention Networks (RNAN) for more efficient and effective SISR. Structurally, our RNAN is a sequential integration of the well-designed Global Context-enhanced Residual Groups (GCRGs), which extracts super-resolved features from coarse to fine. Our GCRG is designed with two novelties. Firstly, the Residual Neuron Attention (RNA) mechanism is proposed in each block of GCRG to reveal the relevance of neurons for better feature representation. Furthermore, the Global Context (GC) block is embedded into RNAN at the end of each GCRG for effectively modeling the global contextual information. Experiments results demonstrate that our RNAN achieves the comparable results with state-of-the-art methods in terms of both quantitative metrics and visual quality, however, with simplified network architecture.

Motivation & Objective

• To address the limitations of simply increasing network depth in super-resolution by focusing on feature relevance and context modeling.
• To improve feature representation in super-resolution networks by explicitly modeling interdependencies between neurons.
• To enhance performance through a lightweight Global Context (GC) block that captures long-range contextual information.
• To achieve superior results with reduced model complexity compared to existing deep networks.

Proposed method

• The network architecture consists of a shallow feature extractor, multiple Global Context-enhanced Residual Groups (GCRGs), an up-sampling module, and a reconstruction layer.
• Each GCRG integrates Residual Neuron Attention (RNA) blocks that recalibrate neuron-wise responses based on learned interdependencies.
• A Global Context (GC) block is appended after each GCRG to model long-range dependencies across the feature map.
• Feature fusion is performed via uniform-spaced concatenation of outputs from different GCRG stages, followed by global residual learning.
• Self-ensemble inference is applied during testing to boost performance by averaging augmented inputs.
• The model is trained end-to-end using L1 loss with Adam optimizer and data augmentation including rotation and flipping.

Experimental results

Research questions

• RQ1 Can modeling neuron-wise feature relationships improve super-resolution performance without increasing network depth?
• RQ2 How does incorporating global contextual information affect the quality of reconstructed high-resolution images?
• RQ3 Can a lightweight attention mechanism outperform deeper or more complex architectures in super-resolution tasks?
• RQ4 What is the trade-off between model complexity and performance in single image super-resolution?

Key findings

• RNAN achieves state-of-the-art PSNR and SSIM scores on Set5, Set14, B100, and Urban100 datasets across ×2, ×3, and ×4 upscaling factors.
• With self-ensemble, RNAN achieves 38.31 dB PSNR on Urban100 at ×4, outperforming EDSR (38.27 dB) and RDN (38.25 dB) with fewer parameters.
• RNAN reduces model parameters to 17.30M, about 2/5 of EDSR (43.10M) and 4/5 of RDN (22.27M), while maintaining competitive performance.
• Visual comparisons show RNAN reconstructs sharper edges and more high-frequency details, such as clearer text and building structures, compared to SRCNN, EDSR, and FSRCNN.
• The performance gap between RNAN and EDSR narrows with higher scaling factors (e.g., 0.04 dB at ×4), yet RNAN delivers significantly better visual quality.
• The combination of RNA and GC blocks enables more effective feature representation, as evidenced by improved SSIM and visual fidelity across diverse image types.