[Paper Review] Domain Adaptation for the Segmentation of Confidential Medical Images

Serban Stan, Mohammad Rostami|arXiv (Cornell University)|Jan 2, 2021

COVID-19 diagnosis using AI4 citations

TL;DR

This paper proposes a novel unsupervised domain adaptation (UDA) method for medical image semantic segmentation that operates without access to source domain data during adaptation, using a Gaussian Mixture Model (GMM) to encode the source domain's internal distribution. The approach achieves competitive performance on cardiac and abdominal MRI/CT segmentation tasks by aligning target features with the learned source distribution, outperforming SOTA methods that require joint source-target access.

ABSTRACT

Convolutional neural networks (CNNs) have led to significant improvements in the semantic segmentation of images. When source and target datasets come from different modalities, CNN performance suffers due to domain shift. In such cases data annotation in the target domain becomes necessary to maintain model performance. To circumvent the re-annotation process, unsupervised domain adaptation (UDA) is proposed to adapt a model to new modalities using solely unlabeled target data. Common UDA algorithms require access to source domain data during adaptation, which may not be feasible in medical imaging due to data sharing restrictions. In this work, we develop an algorithm for UDA where the source domain data is inaccessible during target adaptation. Our approach is based on encoding the source domain information into an internal distribution that is used to guide adaptation in the absence of source samples. We demonstrate the effectiveness of our algorithm by comparing it to state-of-the-art medical image semantic segmentation approaches on two medical image semantic segmentation datasets.

Motivation & Objective

• To address domain shift in medical image segmentation caused by differences in imaging modalities (e.g., MRI vs. CT) when source data cannot be shared due to privacy or security constraints.
• To develop a UDA framework that enables adaptation to a new target domain without requiring access to source domain samples during inference or adaptation.
• To learn and preserve the internal distribution of the source domain using a GMM to guide feature alignment in the target domain.
• To achieve high segmentation accuracy on target-domain medical images while maintaining organ boundary quality and minimizing label shift.
• To demonstrate that indirect domain alignment via learned source distributions can match or exceed performance of methods requiring joint source-target access.

Proposed method

• The method estimates the source domain's internal feature distribution using a Gaussian Mixture Model (GMM) trained on source features before adaptation.
• During adaptation, the model uses the GMM to guide feature alignment between source and target domains by minimizing the distance between target features and the GMM components.
• High-confidence predictions from the source model are used to refine the GMM, improving its representational fidelity to the source distribution.
• The adaptation process employs a self-supervised consistency loss and entropy minimization to refine predictions on unlabeled target data.
• A multi-component GMM per class (with ω = 3 selected as optimal) is used to model complex source distributions, enhancing feature separability.
• The framework uses UMAP visualization to analyze GMM embedding quality and validate class separability across different confidence thresholds (ρ = 0.97).

Experimental results

Research questions

• RQ1 Can a GMM-based internal representation of the source domain effectively guide domain adaptation in medical image segmentation without direct access to source data?
• RQ2 How does the use of high-confidence predictions to refine the GMM impact the quality of domain alignment and segmentation performance?
• RQ3 To what extent does the number of GMM components per class affect segmentation accuracy and boundary precision?
• RQ4 How does the proposed method compare to SOTA UDA methods that require joint source-target access in terms of Dice score and surface distance metrics?
• RQ5 What is the impact of label shift during adaptation, and does it lead to improved or degraded segmentation performance?

Key findings

• The proposed method achieves a mean Dice score of 81.3% on the cardiac MR-to-CT adaptation task, outperforming the 1-SFS baseline (86.2% for AA, 83.5% for LAC, 75.4% for LVC, 70.9% for MYO) and matching or exceeding SOTA methods that require source data access.
• With ω = 3 GMM components per class, the method achieves a 30% reduction in Average Symmetric Surface Distance (ASSD) compared to the 1-SFS baseline, indicating improved boundary localization.
• Label shift analysis shows that over 20% of pixels undergo class migration during adaptation, but the majority of shifts improve segmentation accuracy, particularly between LVC and MYO classes.
• The method demonstrates robustness to high confidence threshold ρ = 0.97, with UMAP visualizations showing improved class separability in the GMM-embedded feature space.
• Ablation studies confirm that using more than one GMM component per class improves performance, with diminishing returns beyond ω = 3, justifying its selection as the optimal balance.
• On the abdominal organ dataset, the method reduces label shift to less than 1% for most classes, with most mislabeled pixels being corrected to 'Ignore' or correctly reallocated, indicating strong generalization.