[论文解读] Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey

Guoping Xu, Xiaxia Wang+3 more|arXiv (Cornell University)|May 2, 2024

Brain Tumor Detection and Classification被引用 10

一句话总结

本综述梳理了跳跃连接和深度神经网络中残差学习在计算机视觉与医学影像领域的起源、发展及影响，概述了 backbone 架构、技术与未来方向。

ABSTRACT

Deep learning has made significant progress in computer vision, specifically in image classification, object detection, and semantic segmentation. The skip connection has played an essential role in the architecture of deep neural networks,enabling easier optimization through residual learning during the training stage and improving accuracy during testing. Many neural networks have inherited the idea of residual learning with skip connections for various tasks, and it has been the standard choice for designing neural networks. This survey provides a comprehensive summary and outlook on the development of skip connections in deep neural networks. The short history of skip connections is outlined, and the development of residual learning in deep neural networks is surveyed. The effectiveness of skip connections in the training and testing stages is summarized, and future directions for using skip connections in residual learning are discussed. Finally, we summarize seminal papers, source code, models, and datasets that utilize skip connections in computer vision, including image classification, object detection, semantic segmentation, and image reconstruction. We hope this survey could inspire peer researchers in the community to develop further skip connections in various forms and tasks and the theory of residual learning in deep neural networks. The project page can be found at https://github.com/apple1986/Residual_Learning_For_Images

研究动机与目标

• Trace the short history and origin of skip connections and residual learning.
• Summarize how skip connections influence training and testing performance in deep nets.
• Review important backbone architectures that inherit residual learning (e.g., ResNet, DenseNet) and their applications in CV/medical imaging.
• Discuss theoretical insights and future directions for residual learning and skip connections across tasks and architectures.

提出的方法

• 梳理从负反馈到残差学习与跳跃连接的历史发展。
• 按跳跃长度、块宽度扩展、判别特征学习、效率以及自注意力整合对残差学习的进展进行分类。
• 总结采用跳跃连接的关键架构与模块（如 ResNet、DenseNet、Inception 变体、ResNeXt、Res2Net、WRN、DenseNet）。
• 讨论增强残差学习的技术（激活顺序、如 SENet/SKNet/ResNeSt 的注意力块、瓶颈重设计、膨胀卷积与可变形卷积）。
• 考察以效率为目标的修改（1x1、分组、深度卷积、随机深度、剪枝、蒸馏）以及使用跳跃连接的自注意力框架（Transformers、ViT、Swin Transformer）。

实验结果

研究问题

• RQ1 跳跃连接在深度神经网络中的起源与演变是什么？
• RQ2 跳跃连接如何影响深度 CV/医学影像模型的优化与泛化？
• RQ3 哪些结构变体（长跳跃与短跳跃、加宽的块、多分支设计）能够最大化性能与效率？
• RQ4 注意力机制与自注意力模型在残差学习中扮演怎样的角色？
• RQ5 哪些资源（数据集、代码、模型）展示了跨 CV 与医学影像的跳跃连接方法？

主要发现

• 跳跃连接/残差学习显著降低优化难度并提升深度网络的最终准确率。
• 长跳跃连接（如 FCN、U-Net）在分割及相关任务中有效融合了细粒度的空间信息与高层语义。
• 加宽残差块和多分支设计（DenseNet、ResNeXt、Res2Net、WRN）可以在提升性能与效率方面超越单纯加深。
• 注意力模块（SENet、SKNet、ResNeSt、CBAM）与残差块结合可提升判别特征学习。
• 先进的卷积技术（膨胀、可变形、深度卷积、分组卷积、1x1）以及随机深度/剪枝策略在不牺牲精度的前提下提高了效率。
• Transformer 与自注意力（ViT、Swin Transformer）利用跳跃连接并仍然是现代 CV 架构与预训练范式的核心。