[Paper Review] Self domain adapted network

Yufan He, Aaron Carass+3 more|arXiv (Cornell University)|Jul 7, 2020

Domain Adaptation and Few-Shot Learning26 references4 citations

TL;DR

This paper proposes Self Domain Adapted Network (SDA-Net), a novel method for rapid, self-supervised domain adaptation in medical imaging that adapts a pre-trained model to a single test subject at inference time without retraining or access to source data. By using autoencoders as alignment metrics and trainable adaptors on pixel, feature, and output spaces, SDA-Net achieves significant performance gains in retinal layer segmentation and T1-to-T2 MRI synthesis with only 30 seconds of adaptation per subject.

ABSTRACT

Domain shift is a major problem for deploying deep networks in clinical practice. Network performance drops significantly with (target) images obtained differently than its (source) training data. Due to a lack of target label data, most work has focused on unsupervised domain adaptation (UDA). Current UDA methods need both source and target data to train models which perform image translation (harmonization) or learn domain-invariant features. However, training a model for each target domain is time consuming and computationally expensive, even infeasible when target domain data are scarce or source data are unavailable due to data privacy. In this paper, we propose a novel self domain adapted network (SDA-Net) that can rapidly adapt itself to a single test subject at the testing stage, without using extra data or training a UDA model. The SDA-Net consists of three parts: adaptors, task model, and auto-encoders. The latter two are pre-trained offline on labeled source images. The task model performs tasks like synthesis, segmentation, or classification, which may suffer from the domain shift problem. At the testing stage, the adaptors are trained to transform the input test image and features to reduce the domain shift as measured by the auto-encoders, and thus perform domain adaptation. We validated our method on retinal layer segmentation from different OCT scanners and T1 to T2 synthesis with T1 from different MRI scanners and with different imaging parameters. Results show that our SDA-Net, with a single test subject and a short amount of time for self adaptation at the testing stage, can achieve significant improvements.

Motivation & Objective

• Address the challenge of domain shift in clinical deep learning, where test data from different scanners or protocols degrade model performance.
• Overcome limitations of existing unsupervised domain adaptation (UDA) methods that require retraining for each target domain and access to target or source data.
• Enable rapid, single-subject adaptation at inference time without extra data or model retraining, especially under data privacy constraints.
• Develop a method that maintains high performance across diverse imaging parameters and scanner types in medical imaging tasks such as segmentation and image synthesis.

Proposed method

• SDA-Net consists of three components: a pre-trained task network (T), pre-trained autoencoders (AEs) for measuring domain alignment, and trainable adaptors for feature and image transformation.
• At inference, adaptors are trained on a single target subject to minimize the autoencoder’s reconstruction error, aligning the target domain with the source domain in pixel, feature, and output spaces.
• The autoencoders act as domain alignment metrics by measuring reconstruction error—lower error indicates closer alignment to the source domain distribution.
• Adaptors are applied at multiple levels: input image (Ax), intermediate features (Ai), and network outputs (Ao), with only Ax and Ai being trained during adaptation.
• The method uses 1×1 convolutions (or 3×3 in ablation) for simplicity and to avoid geometry distortion and feature hallucination common in GAN-based methods.
• Training is performed end-to-end on a single test subject with frozen task and autoencoder networks, enabling fast adaptation in ~30 seconds on a single GPU.

Experimental results

Research questions

• RQ1 Can a deep learning model be rapidly adapted to a single test subject at inference time without access to source data or target labels?
• RQ2 Can autoencoder reconstruction error serve as an effective, differentiable metric for domain alignment in medical image adaptation?
• RQ3 Does minimizing reconstruction error through learnable adaptors on input, features, and outputs lead to better performance than existing UDA methods in low-data regimes?
• RQ4 Can a lightweight adaptor design avoid geometry distortion and feature hallucination while achieving performance comparable to complex GAN-based methods like Cycle-GAN?

Key findings

• SDA-Net achieves significant performance improvements in retinal layer segmentation across different OCT scanners, with a 16.8% relative reduction in MSE compared to the baseline (NA) on the HH dataset.
• For T1-to-T2 MRI synthesis, SDA-Net reduces MSE to 0.168 (±0.040) on the HH dataset and 0.233 (±0.055) on GH, outperforming histogram matching, style transfer, and Cycle-GAN baselines.
• SSIM scores on synthesized T2 images reach 0.693 (±0.039) for HH and 0.658 (±0.043) for GH, indicating high perceptual quality and alignment with source domain features.
• The method achieves comparable or better results than Cycle-GAN with only 30 seconds of adaptation per subject, while avoiding artifacts like geometry shifts and hallucinations common in GANs.
• The Ours-3×3 variant, which uses a 3×3 kernel adaptor, further improves segmentation performance, showing that task-specific adaptor design enhances adaptation.
• Adaptation is fast and efficient—only 5 seconds of inference time without adaptation, and ~30 seconds for adaptation per subject—making it practical for clinical deployment.