[Paper Review] Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns

Hongwei Li, Timo Loehr+4 more|arXiv (Cornell University)|Jan 25, 2020

Domain Adaptation and Few-Shot Learning46 references8 citations

TL;DR

This paper proposes an unsupervised domain adaptation framework for medical image segmentation that improves generalization across domains by adversarially learning disease-specific spatial patterns using few unlabeled target-domain images. By incorporating semantic- and boundary-aware layers and introducing two novel metrics to monitor adaptation without ground truth, the method achieves significant performance gains on cross-centre brain lesion segmentation tasks while preventing overfitting.

ABSTRACT

In medical imaging, the heterogeneity of multi-centre data impedes the applicability of deep learning-based methods and results in significant performance degradation when applying models in an unseen data domain, e.g. a new centreor a new scanner. In this paper, we propose an unsupervised domain adaptation framework for boosting image segmentation performance across multiple domains without using any manual annotations from the new target domains, but by re-calibrating the networks on few images from the target domain. To achieve this, we enforce architectures to be adaptive to new data by rejecting improbable segmentation patterns and implicitly learning through semantic and boundary information, thus to capture disease-specific spatial patterns in an adversarial optimization. The adaptation process needs continuous monitoring, however, as we cannot assume the presence of ground-truth masks for the target domain, we propose two new metrics to monitor the adaptation process, and strategies to train the segmentation algorithm in a stable fashion. We build upon well-established 2D and 3D architectures and perform extensive experiments on three cross-centre brain lesion segmentation tasks, involving multicentre public and in-house datasets. We demonstrate that recalibrating the deep networks on a few unlabeled images from the target domain improves the segmentation accuracy significantly.

Motivation & Objective

• Address domain shift in multi-centre medical imaging where deep learning models degrade due to distribution shifts between scanners and protocols.
• Enable unsupervised domain adaptation for segmentation without requiring manual annotations from the target domain.
• Maintain source domain performance while improving generalization to new, unseen domains through few-shot adaptation.
• Develop reliable training monitoring mechanisms for UDA when ground-truth labels are unavailable in the target domain.

Proposed method

• Introduce a semantic- and boundary-aware layer to encode disease-specific spatial patterns into the feature distribution.
• Apply adversarial training to align the spatial pattern distributions of source and target domains, improving domain generalization.
• Use a segmentation discriminator to enforce realistic and consistent predictions, reducing overfitting.
• Propose two new metrics—variance and difference of predictions—to monitor convergence during UDA without ground truth.
• Fine-tune pre-trained segmentation networks (2D and 3D) on a few unlabeled target images to recalibrate the model.
• Visualize feature distributions via t-SNE to interpret the improvement in separability of lesion and non-lesion features after adaptation.

Experimental results

Research questions

• RQ1 Can adversarial learning of disease-specific spatial patterns improve segmentation performance across different imaging domains without labeled target data?
• RQ2 How can the unsupervised domain adaptation process be effectively monitored and stabilized when no ground-truth labels are available for the target domain?
• RQ3 To what extent do semantic- and boundary-aware layers enhance the model’s ability to learn domain-invariant spatial patterns?
• RQ4 Does the proposed method prevent overfitting on the source domain while improving performance on the target domain?
• RQ5 Can the variance and difference of predictions serve as reliable stopping criteria for UDA training?

Key findings

• The proposed method significantly improves Dice scores on cross-centre brain lesion segmentation tasks, with a 73.9% Dice score on the Utrecht + Amsterdam → Singapore task when all components are used.
• The ablation study shows that incorporating edge and inverse maps leads to a 6.8% improvement in Dice score compared to using only the mask.
• The variance and difference metrics effectively identify the optimal stopping point (epoch 255), corresponding to near-global maximum performance on the target domain.
• Feature visualization confirms that lesion and non-lesion features become more separable after UDA, indicating improved feature discriminability.
• The model maintains stable performance on the source domain and avoids overfitting, as shown by consistent validation metrics over training.
• The method generalizes across both 2D and 3D architectures, demonstrating robustness and transferability across different network designs.