[Paper Review] Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels

Donghyun Kim, Kuniaki Saito+4 more|arXiv (Cornell University)|Mar 18, 2020

Domain Adaptation and Few-Shot Learning43 references33 citations

TL;DR

The paper introduces CDS, a cross-domain self-supervised pre-training approach that yields discriminative and domain-invariant features, enabling effective domain adaptation with only a few labeled source examples and unlabeled target data.

ABSTRACT

Existing unsupervised domain adaptation methods aim to transfer knowledge from a label-rich source domain to an unlabeled target domain. However, obtaining labels for some source domains may be very expensive, making complete labeling as used in prior work impractical. In this work, we investigate a new domain adaptation scenario with sparsely labeled source data, where only a few examples in the source domain have been labeled, while the target domain is unlabeled. We show that when labeled source examples are limited, existing methods often fail to learn discriminative features applicable for both source and target domains. We propose a novel Cross-Domain Self-supervised (CDS) learning approach for domain adaptation, which learns features that are not only domain-invariant but also class-discriminative. Our self-supervised learning method captures apparent visual similarity with in-domain self-supervision in a domain adaptive manner and performs cross-domain feature matching with across-domain self-supervision. In extensive experiments with three standard benchmark datasets, our method significantly boosts performance of target accuracy in the new target domain with few source labels and is even helpful on classical domain adaptation scenarios.

Motivation & Objective

• Motivate learning under domain shift when source labels are scarce, making traditional unsupervised domain adaptation impractical.
• Develop a self-supervised pre-training framework that yields domain-invariant and class-discriminative features without labels.
• Enable effective downstream domain adaptation using few labeled source samples and unlabeled target data.
• Show that CDS pre-trained representations improve target domain accuracy across standard benchmarks.

Proposed method

• Propose Cross-Domain Self-Supervised (CDS) learning with two complementary losses: in-domain instance discrimination and across-domain cross-domain matching.
• Use memory banks for in-domain features from source and target domains to compute in-domain similarity distributions and perform instance discrimination within each domain.
• Introduce cross-domain memory-based matching by computing cross-domain similarities and minimizing the entropy to enforce a positive match across domains and repel unmatched samples.
• Combine the two losses into L_CDS = L_W-INS + L_CDM and update memory banks with a momentum-based update.

Experimental results

Research questions

• RQ1 Can self-supervised pre-training with cross-domain supervision produce features that are both discriminative and domain-invariant in the presence of few source labels?
• RQ2 Does CDS pre-training improve downstream domain adaptation performance over ImageNet pre-trained models across multiple benchmarks and label-scarce settings?
• RQ3 How do in-domain and across-domain self-supervisions contribute to domain alignment and transfer of knowledge from source to target domains?
• RQ4 Is CDS effective across traditional domain adaptation setups as well as the proposed few-source-label scenario?

Key findings

• CDS significantly improves target accuracy over several baselines (including ImageNet pre-training) in Office, Office-Home, and VisDA under 1-shot, 3-shot, and very sparse labeling settings.
• The gains from CDS are larger as the number of labeled source examples decreases, demonstrating robustness to scarce supervision.
• CDS-pretrained representations yield better downstream performance than purely semi-supervised or adversarially trained baselines in domain adaptation tasks.
• Compared to standard self-supervised baselines, CDS achieves substantially higher domain-adaptation-target accuracy, illustrating the benefit of cross-domain alignment.
• Semi-supervised learning on top of CDS can further enhance target accuracy without sacrificing source discriminability.