[Paper Review] Wide Activation for Efficient and Accurate Image Super-Resolution

Jiahui Yu, Yuchen Fan+5 more|arXiv (Cornell University)|Aug 27, 2018

Advanced Image Processing TechniquesComputer Science8 references324 citations

TL;DR

The paper shows that widening feature channels before ReLU in SR residual blocks (WDSR-A and WDSR-B) yields better accuracy at the same parameter/compute budgets, with linear low-rank convolutions enabling even wider activation and weight normalization outperforming batch/normless setups.

ABSTRACT

In this report we demonstrate that with same parameters and computational budgets, models with wider features before ReLU activation have significantly better performance for single image super-resolution (SISR). The resulted SR residual network has a slim identity mapping pathway with wider (\(2 imes\) to \(4 imes\)) channels before activation in each residual block. To further widen activation (\(6 imes\) to \(9 imes\)) without computational overhead, we introduce linear low-rank convolution into SR networks and achieve even better accuracy-efficiency tradeoffs. In addition, compared with batch normalization or no normalization, we find training with weight normalization leads to better accuracy for deep super-resolution networks. Our proposed SR network extit{WDSR} achieves better results on large-scale DIV2K image super-resolution benchmark in terms of PSNR with same or lower computational complexity. Based on WDSR, our method also won 1st places in NTIRE 2018 Challenge on Single Image Super-Resolution in all three realistic tracks. Experiments and ablation studies support the importance of wide activation for image super-resolution. Code is released at: https://github.com/JiahuiYu/wdsr_ntire2018

Motivation & Objective

• Demonstrate that wider activation before ReLU improves SR performance without extra parameters or FLOPs.
• Propose WDSR-A (wide activation) and WDSR-B (wide activation with linear low-rank convolutions) for better accuracy-efficiency.
• Show that weight normalization provides faster convergence and better accuracy than batch normalization or no normalization.
• Evaluate on DIV2K and achieve strong results and NTIRE 2018 wins across tracks.

Proposed method

• Introduce wider activation by expanding features before ReLU within SR residual blocks.
• Maintain the identity mapping pathway width while increasing the pre-activation width to create WDSR-A.
• Develop WDSR-B with linear low-rank convolutions to widen activation further without extra cost.
• Compare with EDSR under identical parameter/compute budgets to isolate the effect of width.
• Advocate weight normalization over batch normalization for training deep SR networks and demonstrate faster convergence.

Experimental results

Research questions

• RQ1 Does widening features before ReLU in SR residual blocks improve accuracy under fixed parameter and compute budgets?
• RQ2 Can efficiency be maintained or improved when widening activation using linear low-rank convolutions?
• RQ3 Is weight normalization more suitable than batch normalization for training deep SR networks?
• RQ4 What gains do WDSR-A and WDSR-B achieve on standard SR benchmarks (e.g., DIV2K) compared to baselines like EDSR?

Key findings

• WDSR-A (2x–4x wider activation) improves DIV2K val PSNR over EDSR at the same budget.
• WDSR-B (6x–9x wider activation) uses linear low-rank convolutions to widen activation with no extra parameters or FLOPs and yields further gains.
• On DIV2K, WDSR variants achieve higher PSNR than the corresponding EDSR baselines at equivalent parameter counts (example results show 33.210 vs. 33.434 PSNR for EDSR and WDSR-B in the 1-residual-block setting).
• Weight normalization accelerates convergence and improves accuracy, while batch normalization leads to instability during testing in SR tasks.