[Paper Review] UNet++: A Nested U-Net Architecture for Medical Image Segmentation

Zongwei Zhou, Md Mahfuzur Rahman Siddiquee+2 more|arXiv (Cornell University)|Jul 18, 2018

Advanced Neural Network Applications9 references270 citations

TL;DR

UNet++ introduces nested dense skip pathways and deep supervision to bridge the semantic gap between encoder and decoder features, achieving higher IoU than U-Net and wide U-Net across multiple medical segmentation tasks.

ABSTRACT

In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in comparison with U-Net and wide U-Net architectures across multiple medical image segmentation tasks: nodule segmentation in the low-dose CT scans of chest, nuclei segmentation in the microscopy images, liver segmentation in abdominal CT scans, and polyp segmentation in colonoscopy videos. Our experiments demonstrate that UNet++ with deep supervision achieves an average IoU gain of 3.9 and 3.4 points over U-Net and wide U-Net, respectively.

Motivation & Objective

• Improve segmentation accuracy for medical images by reducing the semantic gap between encoder and decoder feature maps.
• Introduce re-designed skip pathways with dense connections to enhance gradient flow and feature fusion.
• Leverage deep supervision to enable multi-branch outputs and model pruning without sacrificing performance.

Proposed method

• Propose UNet++ with nested dense skip pathways that progressively enrich encoder features before fusion with decoder features.
• Use dense convolution blocks on skip pathways to bridge semantic gap between encoder and decoder maps.
• Apply deep supervision by adding auxiliary segmentation outputs at multiple semantic levels and combining losses.
• Provide a pruning mechanism via deep supervision to reduce inference time while maintaining accuracy.
• Compare against U-Net and wide U-Net across four datasets to assess performance gains.

Experimental results

Research questions

• RQ1 Does UNet++ achieve higher IoU than U-Net and wide U-Net across diverse medical imaging tasks?
• RQ2 Does deep supervision contribute to segmentation performance and model pruning effectiveness?
• RQ3 How does UNet++ perform in terms of inference speed when pruning is applied?
• RQ4 Which datasets benefit most from the nested skip pathways and deep supervision?

Key findings

• UNet++ without deep supervision achieves an average IoU gain of 2.8 points over U-Net and 3.3 points over wide U-Net.
• UNet++ with deep supervision yields an average IoU gain of 3.9 points over U-Net and 3.4 points over wide U-Net.
• Deep supervision improves segmentation for liver and lung nodule datasets, while its gain is less pronounced for cell nuclei and colon polyp datasets.
• Pruning UNet++ to L3 reduces inference time by 32.2% on average with only a 0.6-point IoU degradation.
• Across four datasets, UNet++ demonstrates superior performance or comparable performance with fewer parameters than the baselines.