[Paper Review] Enforcing Mutual Consistency of Hard Regions for Semi-supervised Medical Image Segmentation.

Yicheng Wu, Zongyuan Ge+5 more|arXiv (Cornell University)|Sep 21, 2021

Advanced Neural Network Applications29 references4 citations

TL;DR

This paper proposes MC-Net+, a semi-supervised medical image segmentation model that enhances learning from unlabeled hard regions by enforcing mutual consistency across multiple decoders with different upsampling strategies. By leveraging statistical discrepancy in decoder outputs to identify uncertainty and applying a mutual consistency constraint between predictions and soft pseudo-labels, MC-Net+ achieves state-of-the-art performance on three public medical imaging datasets.

ABSTRACT

In this paper, we proposed a novel mutual consistency network (MC-Net+) to effectively exploit the unlabeled hard regions for semi-supervised medical image segmentation. The MC-Net+ model is motivated by the observation that deep models trained with limited annotations are prone to output highly uncertain and easily mis-classified predictions in the ambiguous regions (e.g. adhesive edges or thin branches) for the image segmentation task. Leveraging these region-level challenging samples can make the semi-supervised segmentation model training more effective. Therefore, our proposed MC-Net+ model consists of two new designs. First, the model contains one shared encoder and multiple sightly different decoders (i.e. using different up-sampling strategies). The statistical discrepancy of multiple decoders' outputs is computed to denote the model's uncertainty, which indicates the unlabeled hard regions. Second, a new mutual consistency constraint is enforced between one decoder's probability output and other decoders' soft pseudo labels. In this way, we minimize the model's uncertainty during training and force the model to generate invariant and low-entropy results in such challenging areas of unlabeled data, in order to learn a generalized feature representation. We compared the segmentation results of the MC-Net+ with five state-of-the-art semi-supervised approaches on three public medical datasets. Extension experiments with two common semi-supervised settings demonstrate the superior performance of our model over other existing methods, which sets a new state of the art for semi-supervised medical image segmentation.

Motivation & Objective

• To address the challenge of uncertain and misclassified predictions in ambiguous anatomical regions such as thin branches and adhesive edges in semi-supervised medical image segmentation.
• To improve model generalization by effectively exploiting unlabeled hard regions that are typically overlooked in standard semi-supervised learning.
• To reduce model uncertainty in challenging regions through a novel mutual consistency constraint across multiple decoders with different upsampling strategies.
• To develop a training framework that encourages low-entropy, invariant predictions in hard unlabeled regions, leading to more robust feature representations.

Proposed method

• The model uses a shared encoder with multiple decoders that differ only in their upsampling strategies to generate diverse predictions for the same input.
• Model uncertainty is quantified by computing the statistical discrepancy (e.g., variance or divergence) between the outputs of the multiple decoders.
• A mutual consistency constraint is enforced between one decoder’s probability output and the soft pseudo-labels derived from the other decoders’ outputs.
• This constraint minimizes prediction disagreement in hard regions, encouraging the model to produce consistent, low-entropy predictions across all decoders.
• The training objective combines standard segmentation loss with the mutual consistency loss to jointly optimize for accuracy and uncertainty reduction.
• The framework is trained end-to-end and is applicable to various semi-supervised learning settings in medical imaging.

Experimental results

Research questions

• RQ1 Can leveraging uncertainty in unlabeled hard regions improve semi-supervised medical image segmentation performance?
• RQ2 How does enforcing mutual consistency across multiple decoders with different upsampling strategies reduce prediction uncertainty in ambiguous anatomical regions?
• RQ3 Does the proposed mutual consistency constraint lead to more robust and generalizable feature representations compared to standard pseudo-labeling?
• RQ4 To what extent does MC-Net+ outperform existing state-of-the-art methods on public medical image segmentation benchmarks?

Key findings

• MC-Net+ achieves state-of-the-art performance on three public medical image segmentation datasets, outperforming five existing semi-supervised methods.
• The model significantly reduces prediction uncertainty in hard regions such as thin branches and adhesive edges by enforcing mutual consistency across decoders.
• The use of multiple decoders with different upsampling strategies enables better identification of ambiguous regions through statistical discrepancy in their outputs.
• Extension experiments confirm the robustness and generalization of MC-Net+ across two common semi-supervised learning settings.
• The mutual consistency constraint effectively reduces entropy in predictions for unlabeled hard regions, leading to more reliable pseudo-labels.
• The model demonstrates superior generalization by learning invariant representations in challenging anatomical structures, even with limited annotated data.