[Paper Review] The Importance of Skip Connections in Biomedical Image Segmentation

Michal Drozdzal, Eugene Vorontsov+3 more|arXiv (Cornell University)|Aug 14, 2016

Advanced Neural Network Applications19 references112 citations

TL;DR

The paper analyzes the impact of long and short skip connections in very deep Fully Convolutional Networks for biomedical image segmentation, showing that combining both types accelerates training and enables near-state-of-the-art performance on EM data without post-processing.

ABSTRACT

In this paper, we study the influence of both long and short skip connections on Fully Convolutional Networks (FCN) for biomedical image segmentation. In standard FCNs, only long skip connections are used to skip features from the contracting path to the expanding path in order to recover spatial information lost during downsampling. We extend FCNs by adding short skip connections, that are similar to the ones introduced in residual networks, in order to build very deep FCNs (of hundreds of layers). A review of the gradient flow confirms that for a very deep FCN it is beneficial to have both long and short skip connections. Finally, we show that a very deep FCN can achieve near-to-state-of-the-art results on the EM dataset without any further post-processing.

Motivation & Objective

• Investigate how skip connections affect training dynamics in deep FCNs for biomedical image segmentation.
• Extend Residual Networks with an expanding path to perform segmentation.
• Assess the relative benefits of long vs. short skip connections on convergence and performance.

Proposed method

• Extend residual networks to fully convolutional networks with an upsampling path for segmentation.
• Incorporate short skip connections around residual blocks to enable very deep architectures.
• Experiment with different block types (bottleneck, basic, simple) and evaluate with binary cross-entropy and Dice losses.
• Train on EM ISBI 2012 dataset with data augmentation and analyze training/validation curves across architectures.
• Use dropout as an optional regularizer within residual blocks to study its effect on performance and convergence.

Experimental results

Research questions

• RQ1 What is the effect of adding short skip connections on training convergence for very deep FCNs in biomedical segmentation?
• RQ2 Do both long and short skip connections together outperform either type alone in very deep networks?
• RQ3 Can a very deep FCN achieve near-state-of-the-art segmentation on EM data without post-processing?
• RQ4 How do different loss functions (binary cross-entropy vs. Dice loss) interact with skip connections in this setting?

Key findings

• A very deep FCN with both long and short skip connections converges faster and achieves better performance than networks with only one type of skip connection.
• Short skip connections stabilize parameter updates in deep networks and mitigate vanishing gradient effects.
• Networks without skip connections struggle to learn at great depths, while batch normalization enhances trainability and depth.
• Models trained with Dice loss produced visually cleaner segmentations than those trained with binary cross-entropy, with test-time dropout providing implicit model averaging benefits.
• The deep FCN achieved near-state-of-the-art performance on the EM ISBI 2012 dataset without any post-processing."],
• table_headers: [],
• table_rows: []} ;; }json {
• table_headers_empty
• table_rows_empty
• table_headers
• table_rows
• termination
• generated
• notes
• -"} }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } } ,
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }
• } }