[论文解读] Single Image Super-Resolution via Cascaded Multi-Scale Cross Network

Yanting Hu, Xinbo Gao+3 more|arXiv (Cornell University)|Feb 24, 2018

Advanced Image Processing Techniques参考文献 37被引用 42

一句话总结

Introduces a Cascaded Multi-Scale Cross (CMSC) network for single image super-resolution that uses cascaded-subnetworks with multi-scale cross modules and residual-features learning, trained with cascaded-supervision and intermediate predictions to boost reconstruction quality and efficiency.

ABSTRACT

The deep convolutional neural networks have achieved significant improvements in accuracy and speed for single image super-resolution. However, as the depth of network grows, the information flow is weakened and the training becomes harder and harder. On the other hand, most of the models adopt a single-stream structure with which integrating complementary contextual information under different receptive fields is difficult. To improve information flow and to capture sufficient knowledge for reconstructing the high-frequency details, we propose a cascaded multi-scale cross network (CMSC) in which a sequence of subnetworks is cascaded to infer high resolution features in a coarse-to-fine manner. In each cascaded subnetwork, we stack multiple multi-scale cross (MSC) modules to fuse complementary multi-scale information in an efficient way as well as to improve information flow across the layers. Meanwhile, by introducing residual-features learning in each stage, the relative information between high-resolution and low-resolution features is fully utilized to further boost reconstruction performance. We train the proposed network with cascaded-supervision and then assemble the intermediate predictions of the cascade to achieve high quality image reconstruction. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate the superiority of our proposed method over state-of-the-art super-resolution methods.

研究动机与目标

• 推动超分辨率信息流和多尺度特征融合的改进，超越单流架构。
• 提出一个级联网络，按由粗到细的方式逐步细化高分辨率特征。
• 引入多尺度跨（MSC）模块，在不同感受野融合上下文信息。
• 结合残留特征学习与级联监督以提升重建准确性。
• 聚合中间预测以形成最终的高质量高分辨率图像，并提升训练稳定性。

提出的方法

• 构建一个具备特征提取网络、级联子网络和重建网络的 CMSC 架构。
• 在每个级联子网络中，堆叠多个 MSC 模块以融合多尺度信息并学习低分辨率（LR）与高分辨率（HR）特征之间的残留特征。
• 采用类似合并-运行的方案（MSC 模块），具有两条不同卷积核尺寸的分支以捕获多样化的上下文信息。
• 通过在每个子网络中添加恒等跳连来应用残留特征学习，以估计残留的 HR 特征。
• 采用级联监督进行训练，其中中间子网络输出受到监督，最终预测通过加权平均进行组装。

实验结果

研究问题

• RQ1 级联的多尺度融合网络是否能在超分辨率性能上优于传统的单流深度超分辨率模型？
• RQ2 MSC 模块能否有效融合多尺度信息并提升深度超分辨率网络的信息流？
• RQ3 在每个级联阶段中的残留特征学习是否能提升收敛性和重建质量？
• RQ4 级联监督和中间预测组装是否比单一最终监督带来更好的超分辨率效果？

主要发现

• CMSC 在执行时间相对较低的情况下实现了更优的性能（如经验结果所示）。
• MSC 模块在不同感受野下有效融合多尺度上下文信息并改善信息流。
• 具有多阶段和残留特征学习的级联结构以粗到细的方式逐步细化 HR 特征。
• 级联监督与中间预测组装进一步提升 SR 的准确性。
• 该方法在标准 SR 基准测试上得到验证，量化提升相较于竞争方法。