[Paper Review] Self-training with dual uncertainty for semi-supervised medical image segmentation

Zhanhong Qiu, Haitao Gan+3 more|arXiv (Cornell University)|Apr 10, 2023

Radiomics and Machine Learning in Medical ImagingMedicine3 citations

TL;DR

This paper proposes a self-training framework with dual uncertainty estimation—sample-level and pixel-level—to improve semi-supervised medical image segmentation. By leveraging model prediction variance across multiple checkpoints for sample-level uncertainty and output differences from two decoders with distinct upsampling methods for pixel-level uncertainty, the method selectively re-trains hard unlabeled samples and refines pseudo-labels, achieving state-of-the-art performance on the ACDC and Prostate datasets under identical settings.

ABSTRACT

In the field of semi-supervised medical image segmentation, the shortage of labeled data is the fundamental problem. How to effectively learn image features from unlabeled images to improve segmentation accuracy is the main research direction in this field. Traditional self-training methods can partially solve the problem of insufficient labeled data by generating pseudo labels for iterative training. However, noise generated due to the model's uncertainty during training directly affects the segmentation results. Therefore, we added sample-level and pixel-level uncertainty to stabilize the training process based on the self-training framework. Specifically, we saved several moments of the model during pre-training, and used the difference between their predictions on unlabeled samples as the sample-level uncertainty estimate for that sample. Then, we gradually add unlabeled samples from easy to hard during training. At the same time, we added a decoder with different upsampling methods to the segmentation network and used the difference between the outputs of the two decoders as pixel-level uncertainty. In short, we selectively retrained unlabeled samples and assigned pixel-level uncertainty to pseudo labels to optimize the self-training process. We compared the segmentation results of our model with five semi-supervised approaches on the public 2017 ACDC dataset and 2018 Prostate dataset. Our proposed method achieves better segmentation performance on both datasets under the same settings, demonstrating its effectiveness, robustness, and potential transferability to other medical image segmentation tasks. Keywords: Medical image segmentation, semi-supervised learning, self-training, uncertainty estimation

Motivation & Objective

• To address the critical challenge of limited annotated medical image data in segmentation tasks.
• To stabilize the self-training process in semi-supervised learning by reducing noise from model uncertainty.
• To improve segmentation accuracy by selectively retraining hard unlabeled samples based on uncertainty estimates.
• To enhance pseudo-label quality through pixel-level uncertainty estimation using dual decoders.
• To demonstrate generalization and robustness across diverse medical imaging datasets.

Proposed method

• Sample-level uncertainty is estimated as the variance of model predictions across multiple checkpoints during pre-training on the same unlabeled sample.
• Unlabeled samples are gradually introduced into training in order of increasing sample-level uncertainty, from easy to hard.
• Pixel-level uncertainty is computed as the difference between outputs from two decoders using different upsampling strategies (e.g., transposed convolution vs. deconvolution).
• Pseudo-labels are refined using pixel-level uncertainty, assigning higher confidence to regions with lower uncertainty.
• The model iteratively improves through self-training, retraining only the most uncertain samples to reduce error propagation.
• The framework integrates uncertainty-aware pseudo-labeling into a standard self-training pipeline without architectural overhauls.

Experimental results

Research questions

• RQ1 Can sample-level uncertainty estimation improve the reliability of pseudo-labels in semi-supervised medical image segmentation?
• RQ2 Can pixel-level uncertainty derived from dual decoders enhance pseudo-label quality and reduce noise?
• RQ3 Does selective retraining of hard unlabeled samples based on dual uncertainty improve segmentation performance?
• RQ4 How does the proposed method compare to existing self-training approaches on benchmark medical imaging datasets?
• RQ5 To what extent is the method robust and transferable across different medical image segmentation tasks?

Key findings

• The proposed method achieved state-of-the-art performance on the 2017 ACDC dataset under the same training protocol as baseline methods.
• On the 2018 Prostate dataset, the method outperformed five existing semi-supervised approaches in segmentation accuracy.
• The integration of dual uncertainty significantly reduced error propagation from noisy pseudo-labels during self-training.
• The model demonstrated improved robustness by selectively retraining only the most uncertain samples, avoiding degradation from low-quality pseudo-labels.
• The method showed strong generalization potential, with consistent performance gains across two distinct medical imaging tasks.
• The use of multiple model checkpoints and dual decoders provided reliable uncertainty estimates without requiring additional training or inference cost.