[Paper Review] Normalization in Training U-Net for 2D Biomedical Semantic Segmentation

Xiao-Yun Zhou, Guang‐Zhong Yang|arXiv (Cornell University)|Sep 11, 2018

Advanced Neural Network Applications22 references4 citations

TL;DR

This paper evaluates four normalization techniques—Batch Normalization (BN), Instance Normalization (IN), Layer Normalization (LN), and Group Normalization (GN)—in training U-Net for 2D biomedical semantic segmentation. It finds that GN with a large group number or IN improves model generalization and segmentation accuracy more effectively than BN, especially on small-batch or patient-specific data, due to better handling of internal covariate shift in medical imaging.

ABSTRACT

2D biomedical semantic segmentation is important for robotic vision in surgery. Segmentation methods based on Deep Convolutional Neural Network (DCNN) can out-perform conventional methods in terms of both accuracy and levels of automation. One common issue in training a DCNN for biomedical semantic segmentation is the internal covariate shift where the training of convolutional kernels is encumbered by the distribution change of input features, hence both the training speed and performance are decreased. Batch Normalization (BN) is the first proposed method for addressing internal covariate shift and is widely used. Instance Normalization (IN) and Layer Normalization (LN) have also been proposed. Group Normalization (GN) is proposed more recently and has not yet been applied to 2D biomedical semantic segmentation, however, no specific validations on GN were given. Most DCNNs for biomedical semantic segmentation adopt BN as the normalization method by default, without reviewing its performance. In this paper, four normalization methods - BN, IN, LN and GN are compared in details, specifically for 2D biomedical semantic segmentation. U-Net is adopted as the basic DCNN structure. Three datasets regarding the Right Ventricle (RV), aorta, and Left Ventricle (LV) are used for the validation. The results show that detailed subdivision of the feature map, i.e. GN with a large group number or IN, achieves higher accuracy. This accuracy improvement mainly comes from better model generalization. Codes are uploaded and maintained at Xiao-Yun Zhou's Github.

Motivation & Objective

• To investigate the impact of different normalization techniques on U-Net training for 2D biomedical semantic segmentation.
• To address the challenge of internal covariate shift in medical image segmentation, where feature distribution shifts during training degrade model performance.
• To evaluate whether alternative normalization methods like GN, IN, and LN outperform the widely used Batch Normalization (BN) in terms of segmentation accuracy and generalization.
• To validate the effectiveness of normalization methods across diverse anatomical structures (RV, aorta, LV) and patient-specific data with varying intensity distributions.
• To guide future design of normalization layers in medical image deep learning by identifying optimal strategies for generalization and robustness.

Proposed method

• Trained U-Net with four normalization methods: Batch Normalization (BN), Instance Normalization (IN), Layer Normalization (LN), and Group Normalization (GN) with varying group numbers (GN4, GN8, GN16).
• Applied normalization after each convolutional layer in the U-Net encoder and decoder to stabilize feature distributions and reduce internal covariate shift.
• Used three public 2D cardiac MRI/CT datasets for Right Ventricle (RV), aorta, and Left Ventricle (LV) segmentation to evaluate performance across different anatomical structures.
• Evaluated models using the Dice Similarity Coefficient (DSC) as the primary metric, with repeated training runs to assess stability and generalization.
• Compared inference behavior using both training-time statistics (TrainI) and test-time statistics (TestI) for BN to assess robustness under distribution shifts.
• Conducted ablation studies on batch size effects and normalization method performance across multiple random seeds to assess training stability.

Experimental results

Research questions

• RQ1 Does Group Normalization (GN) with high group count or Instance Normalization (IN) outperform Batch Normalization (BN) in 2D biomedical semantic segmentation using U-Net?
• RQ2 How do different normalization methods affect model generalization, especially on patient-specific data with varying image intensity distributions?
• RQ3 What is the impact of batch size on the performance of normalization techniques in U-Net training for medical image segmentation?
• RQ4 Does using test-time statistics (TestI) instead of training-time statistics (TrainI) during inference improve BN performance in medical segmentation tasks?
• RQ5 Can normalization techniques mitigate the negative effects of internal covariate shift in deep U-Net architectures trained from scratch on medical images?

Key findings

• Group Normalization (GN) with a large group number (e.g., GN16) achieved the highest mean DSC of 0.8091 on the aorta dataset, outperforming BN (0.8274) and LN (0.8189) on the LV dataset.
• Instance Normalization (IN) achieved a mean DSC of 0.7035 on the RV dataset, significantly outperforming BN (0.6436) and showing the best performance on this challenging, low-contrast structure.
• Normalization methods, especially GN and IN, improved segmentation accuracy most on patients with initially low accuracy (e.g., patient 31 for RV, patient 15 for aorta), indicating enhanced generalization.
• The standard deviation across six independent training runs was highest for LN (0.0562) and lowest for GN8 (0.0097), indicating GN with moderate group size offers better training stability.
• Small batch sizes outperformed large batch sizes in terms of segmentation accuracy, suggesting that BN's performance is sensitive to batch size in medical imaging.
• Using test-time statistics (TestI) instead of training-time statistics (TrainI) during inference improved BN performance, indicating that BN's statistics are sensitive to batch distribution shifts.