[Paper Review] Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian+3 more|arXiv (Cornell University)|Feb 24, 2018

Advanced Image Processing TechniquesComputer Science34 references247 citations

TL;DR

The paper introduces a very deep Residual Dense Network (RDN) that fully exploits hierarchical features from the original low-resolution image through residual dense blocks, achieving state-of-the-art results on multiple degradation models for image super-resolution.

ABSTRACT

A very deep convolutional neural network (CNN) has recently achieved great success for image super-resolution (SR) and offered hierarchical features as well. However, most deep CNN based SR models do not make full use of the hierarchical features from the original low-resolution (LR) images, thereby achieving relatively-low performance. In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers. Specifically, we propose residual dense block (RDB) to extract abundant local features via dense connected convolutional layers. RDB further allows direct connections from the state of preceding RDB to all the layers of current RDB, leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is then used to adaptively learn more effective features from preceding and current local features and stabilizes the training of wider network. After fully obtaining dense local features, we use global feature fusion to jointly and adaptively learn global hierarchical features in a holistic way. Extensive experiments on benchmark datasets with different degradation models show that our RDN achieves favorable performance against state-of-the-art methods.

Motivation & Objective

• Motivate improved image SR by leveraging hierarchical features from the original LR image.
• Propose a novel building block (RDB) with contiguous memory and local feature fusion to enable very deep, wide networks.
• Develop a global feature fusion framework to jointly fuse multi-level features from all RDBs in LR space.
• Demonstrate robustness of RDN across multiple degradation models and real-world images.
• Provide implementation details and ablation studies to justify design choices.

Proposed method

• Introduce residual dense block (RDB) with dense connections, local feature fusion (LFF) via a 1x1 conv, and local residual learning (LRL) to form a contiguous memory (CM) mechanism.
• Stack multiple RDBs to form a residual dense network (RDN) with shallow feature extraction, dense feature fusion (DFF) including global feature fusion (GFF) and global residual learning (GRL), and an upsampling net (UPNet) using ESPCN.
• Extract features in the LR space, fuse them globally to form dense features, then upsample to HR via ESPCN and a final conv layer.
• Train with L1 loss, using DIV2K for training and standard SR benchmarks (Set5, Set14, B100, Urban100, Manga109) for evaluation under several degradation models (BI, BD, DN).
• Perform ablations to assess the contributions of contiguous memory (CM), local residual learning (LRL), and global feature fusion (GFF).

Experimental results

Research questions

• RQ1 Can hierarchical features from all layers in the LR space improve SR performance compared to using only the final LR features?
• RQ2 Do contiguous memory, local feature fusion, and global feature fusion synergistically enhance training stability and SR accuracy in a very deep network?
• RQ3 How does RDN perform across different degradation models (bicubic, blur-down, and noisy-down) and on real-world images?
• RQ4 What is the impact of network depth (D), per-block layers (C), and growth rate (G) on SR performance?
• RQ5 Is a global fusion of multi-level LR features superior to local-only dense connections for SR?

Key findings

• RDN achieves superior average PSNR/SSIM on multiple datasets compared to state-of-the-art methods under BI degradation, especially at scale ×2.
• Ablations show that CM, LRL, and GFF each improve performance, with all three together delivering the best results.
• Global feature fusion and global residual learning enable effective combination of shallow and deep features, improving training stability and SR quality.
• RDN consistently outperforms competing methods across Set5, Set14, B100, Urban100, and Manga109, particularly at smaller scales (×2).
• RDN demonstrates robustness to BD (blur-down) and DN (noisy-down) degradations and yields sharper edges and preserved details in qualitative comparisons.
• Self-ensembling (RDN+) provides additional SR gains in BI degradation tests.
• Real-world images with unknown degradation show RDN yields sharper edges and finer details than several baselines.