[论文解读] Single Image Super-resolution via a Lightweight Residual Convolutional Neural Network

Yudong Liang, Ze Yang+4 more|arXiv (Cornell University)|Mar 23, 2017

Advanced Image Processing Techniques参考文献 37被引用 13

一句话总结

该论文提出了一种用于单图像超分辨率的轻量化残差卷积神经网络，通过跳跃连接缓解梯度消失问题，并采用一种新颖的逐步变化网络深度和宽度的策略，在保持高性能的同时减少参数量。该方法在基准数据集上实现了最先进的PSNR和SSIM结果，同时提升了训练稳定性和视觉质量。

ABSTRACT

Recent years have witnessed great success of convolutional neural network (CNN) for various problems both in low and high level visions. Especially noteworthy is the residual network which was originally proposed to handle high-level vision problems and enjoys several merits. This paper aims to extend the merits of residual network, such as skip connection induced fast training, for a typical low-level vision problem, i.e., single image super-resolution. In general, the two main challenges of existing deep CNN for supper-resolution lie in the gradient exploding/vanishing problem and large numbers of parameters or computational cost as CNN goes deeper. Correspondingly, the skip connections or identity mapping shortcuts are utilized to avoid gradient exploding/vanishing problem. In addition, the skip connections have naturally centered the activation which led to better performance. To tackle with the second problem, a lightweight CNN architecture which has carefully designed width, depth and skip connections was proposed. In particular, a strategy of gradually varying the shape of network has been proposed for residual network. Different residual architectures for image super-resolution have also been compared. Experimental results have demonstrated that the proposed CNN model can not only achieve state-of-the-art PSNR and SSIM results for single image super-resolution but also produce visually pleasant results. This paper has extended the mmm 2017 oral conference paper with a considerable new analyses and more experiments especially from the perspective of centering activations and ensemble behaviors of residual network.

研究动机与目标

• 为解决深度卷积神经网络在单图像超分辨率（SISR）任务中面临的梯度消失和高计算成本问题。
• 通过利用跳跃连接和恒等快捷连接，提升极深网络的训练稳定性和收敛性。
• 通过一种新型网络架构实现深度和宽度的逐步变化，以降低模型复杂度和参数量。
• 研究网络“形状”对性能的影响，尤其关注残差结构与普通网络结构的差异。
• 证明残差网络可被高效设计用于轻量化SISR任务，并在多种放大倍数下具备强大的泛化能力。

提出的方法

• 提出一种深层残差CNN，其残差单元以分层方式堆叠，每个残差单元包含两层卷积层。
• 提出一种在各层中逐步变化特征图数量的策略（例如，从16到64再回到16），以平衡表征能力和参数效率。
• 通过跳跃连接（恒等快捷连接）稳定训练过程，并缓解梯度消失问题，从而支持更深的网络结构。
• 通过残差连接构建浅层路径的集成结构，增强对网络架构变化的鲁棒性。
• 通过在单一网络中联合训练多个放大倍数（×2、×3、×4）来提升模型泛化能力。
• 采用残差学习策略，通过预测高频分量（HR - LR）而非直接回归完整HR图像，使激活值居中，从而改善收敛性。

实验结果

研究问题

• RQ1 残差网络的架构形状在单图像超分辨率任务中如何影响性能？
• RQ2 参数量更少的轻量化残差CNN是否能在SISR任务中超越更深或更宽的基线模型？
• RQ3 跳跃连接和残差学习在深度SISR网络中如何影响梯度稳定性和收敛性？
• RQ4 与单尺度训练相比，联合训练多个放大倍数在多大程度上提升模型泛化能力？
• RQ5 残差网络的集成特性在多大程度上增强了对架构变化的鲁棒性？

主要发现

• 所提出的轻量化残差网络在×2放大下于Set14数据集上实现了32.94 dB的PSNR，优于其他形状各异的残差模型。
• 在Set5数据集上×2放大时，该模型实现了37.52 dB的PSNR，与先前工作中报告的最先进结果相当或略优。
• 具有逐步变化深度和宽度的网络在不同形状下均表现出一致的性能，表明其对架构设计具有鲁棒性。
• 联合训练多个放大倍数略微提升了性能，尤其在较大放大倍数（×3和×4）下表现更优，表明泛化能力增强。
• VDSR基线模型对形状变化更为敏感，PSNR在32.50至32.85 dB之间波动，而残差模型则稳定保持在约32.9 dB。
• 残差网络的集成特性结合跳跃连接，使其在不同网络配置下均表现出更稳定和一致的性能。