[論文レビュー] Residual Dense Network for Image Super-Resolution

Yulun Zhang, Yapeng Tian+3 more|arXiv (Cornell University)|Feb 24, 2018

Advanced Image Processing Techniques参考文献 34被引用数 247

ひとこと要約

この論文は、元の低解像度画像から残差密結合ブロックを通じて階層的特徴を完全に活用する非常に深い Residual Dense Network (RDN) を導入し、画像超解像の複数の劣化モデルで最先端の結果を達成する。

ABSTRACT

A very deep convolutional neural network (CNN) has recently achieved great success for image super-resolution (SR) and offered hierarchical features as well. However, most deep CNN based SR models do not make full use of the hierarchical features from the original low-resolution (LR) images, thereby achieving relatively-low performance. In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers. Specifically, we propose residual dense block (RDB) to extract abundant local features via dense connected convolutional layers. RDB further allows direct connections from the state of preceding RDB to all the layers of current RDB, leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is then used to adaptively learn more effective features from preceding and current local features and stabilizes the training of wider network. After fully obtaining dense local features, we use global feature fusion to jointly and adaptively learn global hierarchical features in a holistic way. Extensive experiments on benchmark datasets with different degradation models show that our RDN achieves favorable performance against state-of-the-art methods.

研究の動機と目的

• Motivate improved image SR by leveraging hierarchical features from the original LR image.
• Propose a novel building block (RDB) with contiguous memory and local feature fusion to enable very deep, wide networks.
• Develop a global feature fusion framework to jointly fuse multi-level features from all RDBs in LR space.
• Demonstrate robustness of RDN across multiple degradation models and real-world images.
• Provide implementation details and ablation studies to justify design choices.

提案手法

• Introduce residual dense block (RDB) with dense connections, local feature fusion (LFF) via a 1x1 conv, and local residual learning (LRL) to form a contiguous memory (CM) mechanism.
• Stack multiple RDBs to form a residual dense network (RDN) with shallow feature extraction, dense feature fusion (DFF) including global feature fusion (GFF) and global residual learning (GRL), and an upsampling net (UPNet) using ESPCN.
• Extract features in the LR space, fuse them globally to form dense features, then upsample to HR via ESPCN and a final conv layer.
• Train with L1 loss, using DIV2K for training and standard SR benchmarks (Set5, Set14, B100, Urban100, Manga109) for evaluation under several degradation models (BI, BD, DN).
• Perform ablations to assess the contributions of contiguous memory (CM), local residual learning (LRL), and global feature fusion (GFF).

実験結果

リサーチクエスチョン

• RQ1 Can hierarchical features from all layers in the LR space improve SR performance compared to using only the final LR features?
• RQ2 Do contiguous memory, local feature fusion, and global feature fusion synergistically enhance training stability and SR accuracy in a very deep network?
• RQ3 How does RDN perform across different degradation models (bicubic, blur-down, and noisy-down) and on real-world images?
• RQ4 What is the impact of network depth (D), per-block layers (C), and growth rate (G) on SR performance?
• RQ5 Is a global fusion of multi-level LR features superior to local-only dense connections for SR?

主な発見

• RDN achieves superior average PSNR/SSIM on multiple datasets compared to state-of-the-art methods under BI degradation, especially at scale ×2.
• Ablations show that CM, LRL, and GFF each improve performance, with all three together delivering the best results.
• Global feature fusion and global residual learning enable effective combination of shallow and deep features, improving training stability and SR quality.
• RDN consistently outperforms competing methods across Set5, Set14, B100, Urban100, and Manga109, particularly at smaller scales (×2).
• RDN demonstrates robustness to BD (blur-down) and DN (noisy-down) degradations and yields sharper edges and preserved details in qualitative comparisons.
• Self-ensembling (RDN+) provides additional SR gains in BI degradation tests.
• Real-world images with unknown degradation show RDN yields sharper edges and finer details than several baselines.