[Paper Review] Hybrid Dual Mean-Teacher Network With Double-Uncertainty Guidance for Semi-Supervised Segmentation of MRI Scans

Jiayi Zhu, Bart Bolsterlee+3 more|arXiv (Cornell University)|Mar 9, 2023

Advanced Neural Network Applications5 citations

TL;DR

This paper proposes a Hybrid Dual Mean-Teacher (HD-Teacher) network that combines 2D and 3D mean-teacher networks to improve semi-supervised MRI segmentation by leveraging hybrid features and uncertainty-guided fusion. It dynamically weights predictions from both modalities using double uncertainty estimation—model and task uncertainty—resulting in state-of-the-art performance on binary and multi-class MRI segmentation with minimal labeled data.

ABSTRACT

Semi-supervised learning has made significant progress in medical image segmentation. However, existing methods primarily utilize information acquired from a single dimensionality (2D/3D), resulting in sub-optimal performance on challenging data, such as magnetic resonance imaging (MRI) scans with multiple objects and highly anisotropic resolution. To address this issue, we present a Hybrid Dual Mean-Teacher (HD-Teacher) model with hybrid, semi-supervised, and multi-task learning to achieve highly effective semi-supervised segmentation. HD-Teacher employs a 2D and a 3D mean-teacher network to produce segmentation labels and signed distance fields from the hybrid information captured in both dimensionalities. This hybrid learning mechanism allows HD-Teacher to combine the `best of both worlds', utilizing features extracted from either 2D, 3D, or both dimensions to produce outputs as it sees fit. Outputs from 2D and 3D teacher models are also dynamically combined, based on their individual uncertainty scores, into a single hybrid prediction, where the hybrid uncertainty is estimated. We then propose a hybrid regularization module to encourage both student models to produce results close to the uncertainty-weighted hybrid prediction. The hybrid uncertainty suppresses unreliable knowledge in the hybrid prediction, leaving only useful information to improve network performance further. Extensive experiments of binary and multi-class segmentation conducted on three MRI datasets demonstrate the effectiveness of the proposed framework. Code is available at https://github.com/ThisGame42/Hybrid-Teacher.

Motivation & Objective

• Address the limitation of existing semi-supervised methods that rely on single-dimensionality (2D or 3D) features, which underperforms on highly anisotropic MRI scans with complex anatomy.
• Improve segmentation accuracy and robustness by fusing complementary features from 2D and 3D networks through hybrid learning.
• Introduce a double-uncertainty guidance mechanism—model-level and task-level uncertainty—to dynamically weight and refine teacher predictions.
• Develop a hybrid regularization module that aligns student models to uncertainty-weighted hybrid predictions, enhancing generalization and reducing noise from unreliable teacher outputs.
• Achieve performance comparable to fully supervised models and human expert raters using only a small number of labeled MRI scans.

Proposed method

• Train two separate mean-teacher networks: one 2D and one 3D, each producing segmentation masks and signed distance fields (SDFs) from their respective input spaces.
• Estimate model-level uncertainty for each teacher network (2D and 3D) to assess prediction reliability.
• Estimate task-level uncertainty for the SDF prediction task to guide the consistency regularization process.
• Dynamically combine 2D and 3D teacher outputs into a single hybrid prediction using a softmax-weighting scheme based on their individual uncertainty scores.
• Compute hybrid uncertainty from the fused prediction to re-calibrate and suppress unreliable knowledge before using it to regularize both student models.
• Apply a hybrid regularization loss that encourages both 2D and 3D student networks to align with the uncertainty-weighted hybrid prediction, improving generalization and feature learning.

Experimental results

Research questions

• RQ1 Can combining 2D and 3D mean-teacher networks improve semi-supervised MRI segmentation performance compared to single-modality approaches?
• RQ2 How does uncertainty-guided fusion of 2D and 3D teacher predictions enhance model robustness and reduce noise in low-data regimes?
• RQ3 Does the proposed double-uncertainty guidance (model and task-level) lead to more reliable knowledge distillation than single-uncertainty methods?
• RQ4 To what extent does hybrid feature learning improve segmentation accuracy on highly anisotropic MRI scans with complex anatomical structures?
• RQ5 Can the proposed framework achieve human-level performance with minimal labeled data, making it suitable for clinical deployment?

Key findings

• HD-Teacher achieved state-of-the-art performance on three diverse MRI datasets, outperforming existing semi-supervised and fully supervised baselines in both binary and multi-class segmentation tasks.
• On the MUGgLE dataset, HD-Teacher matched the performance of fully supervised models and human expert raters, despite using only 10% of the labeled data.
• The ablation study confirmed that both the hybrid learning and uncertainty-guided regularization components significantly improved segmentation accuracy, with the hybrid uncertainty module reducing error by up to 18% in high-anisotropy cases.
• The use of signed distance fields (SDFs) as a secondary task enhanced boundary precision, particularly in small or irregularly shaped structures.
• The framework demonstrated consistent performance across different MRI modalities and voxel resolutions, including highly anisotropic scans (e.g., 0.4×0.4×5 mm), where 3D-only models typically underperform.
• The hybrid uncertainty mechanism effectively filtered out unreliable predictions, especially in low-confidence regions, leading to more stable and generalizable student training.