[Paper Review] Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation

Jian Liang, Dapeng Hu+1 more|arXiv (Cornell University)|Feb 19, 2020

Domain Adaptation and Few-Shot Learning324 citations

TL;DR

The paper introduces SHOT, a framework for unsupervised domain adaptation that uses a frozen source classifier and a target-specific encoder, along with self-supervised pseudo-labeling, to adapt without accessing source data. SHOT achieves state-of-the-art results across various DA settings.

ABSTRACT

Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model, making them risky and inefficient for decentralized private data. This work tackles a practical setting where only a trained source model is available and investigates how we can effectively utilize such a model without source data to solve UDA problems. We propose a simple yet generic representation learning framework, named \\emph{Source HypOthesis Transfer} (SHOT). SHOT freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis. To verify its versatility, we evaluate SHOT in a variety of adaptation cases including closed-set, partial-set, and open-set domain adaptation. Experiments indicate that SHOT yields state-of-the-art results among multiple domain adaptation benchmarks.

Motivation & Objective

• Motivate a practical unsupervised DA setting where only a trained source model is available, preserving source data privacy.
• Propose a generic representation learning framework (SHOT) that aligns target representations to the source hypothesis without source data.
• Improve target feature learning via information maximization and self-supervised pseudo-labeling.
• Evaluate SHOT across closed-set, partial-set, and open-set DA benchmarks to demonstrate versatility and performance.
• Explore network architecture choices (weight normalization, batch normalization, label smoothing) to boost adaptation.

Proposed method

• Freeze the source classifier (hypothesis) h_s and learn a target feature encoder g_t to produce target representations suitable for the source hypothesis.
• Optimize an information maximization (IM) objective comprising entropy minimization and diversity promotion to drive target outputs toward one-hot yet diverse predictions.
• Introduce self-supervised pseudo-labeling based on target domain prototypes to refine target representations and reduce reliance on noisy source-driven labels.
• Incorporate pseudo-label supervision with a balancing term to combine IM losses and pseudo-label guidance (β parameter).
• Use weight normalization, batch normalization, and label smoothing within the network to enhance adaptation.
• Formulate SHOT as an end-to-end framework that can extend to partial/open-set DA settings.

Experimental results

Research questions

• RQ1 Can a trained source model be effectively used for unsupervised domain adaptation without access to the source data?
• RQ2 How can target feature learning be guided to align with the source hypothesis without sharing source data?
• RQ3 Does information maximization plus self-supervised pseudo-labeling yield robust cross-domain representations across closed-, partial-, and open-set DA?
• RQ4 What architectural choices (BN, WN, LS) further boost SHOT’s adaptation performance?

Key findings

• SHOT-IM and SHOT consistently outperform source-model-only baselines across multiple benchmarks.
• On the Office-Home dataset, SHOT improves average accuracy to 71.8% in closed-set DA and to 79.3% in open-set/partial settings compared to prior methods.
• For digits, Office, and VisDA-C tasks, SHOT achieves state-of-the-art or competitive results, with SHOT often surpassing SHOT-IM due to self-supervised pseudo-labeling.
• Self-supervised pseudo-labeling yields stronger performance than naive pseudo-labeling, and combining L_ent with L_div generally provides the best results.
• In ablations, BN, WN, and LS contribute to performance, and their combination with SHOT-IM yields the best overall results.
• Table-like results show SHOT achieves high averages (e.g., SHOT full on Office-Home surpasses prior methods by notable margins) and strong per-task gains.