[Paper Review] Unsupervised Domain Adaptation through Self-Supervision

Yu Sun, Eric Tzeng+2 more|arXiv (Cornell University)|Sep 26, 2019

Domain Adaptation and Few-Shot Learning64 references168 citations

TL;DR

The paper proposes aligning source and target domains by jointly training a main task with multiple self-supervised auxiliary tasks on both domains, avoiding adversarial minimax optimization and achieving state-of-the-art results on several benchmarks.

ABSTRACT

This paper addresses unsupervised domain adaptation, the setting where labeled training data is available on a source domain, but the goal is to have good performance on a target domain with only unlabeled data. Like much of previous work, we seek to align the learned representations of the source and target domains while preserving discriminability. The way we accomplish alignment is by learning to perform auxiliary self-supervised task(s) on both domains simultaneously. Each self-supervised task brings the two domains closer together along the direction relevant to that task. Training this jointly with the main task classifier on the source domain is shown to successfully generalize to the unlabeled target domain. The presented objective is straightforward to implement and easy to optimize. We achieve state-of-the-art results on four out of seven standard benchmarks, and competitive results on segmentation adaptation. We also demonstrate that our method composes well with another popular pixel-level adaptation method.

Motivation & Objective

• Motivate unsupervised domain adaptation where labeled data exists only in the source domain and unlabeled data is available in the target domain.
• Propose using jointly trained self-supervised tasks on both domains to align representations while preserving discriminability on the source.
• Develop a simple, stable training objective that avoids adversarial/discrepancy minimax optimization.
• Show that combining self-supervised tasks yields strong performance and can complement pixel-level adaptation methods.

Proposed method

• Introduce K self-supervised auxiliary tasks that generate labels from data (e.g., rotation, flip, location) and train them alongside the main supervised task.
• Use a shared encoder φ with task-specific heads hk for each self-supervised task and the main task, optimizing L0 with respect to h0 and φ, and Li for i=1..K with respect to hi and φ.
• Formulate the objective as a multi-task learning problem: minimize L0(S;φ,h0) + sumk=1..K Lk(S,T;φ,hk), where Sk are labeled source samples and Tk are unlabeled source/target samples augmented through self-supervised transformations.
• Generates self-supervised labels from both source and target data ensuring alignment along directions relevant to each task (rotation, location, flip).
• Propose a simple heuristic for hyper-parameter tuning and early stopping using the mean feature-space domain discrepancy D(S′,T′;φ) and main task error on the source validation set to guide training without target labels.

Experimental results

Research questions

• RQ1 Can jointly trained self-supervised tasks on both source and target domains align the representations sufficiently to enable good target-domain performance without target labels?
• RQ2 Which self-supervised tasks (e.g., rotation, location, flipping) best promote domain alignment for different application domains?
• RQ3 Is it possible to avoid adversarial/minimax optimization in unsupervised domain adaptation while achieving competitive results?
• RQ4 How can hyper-parameter tuning and early stopping be effectively performed without labeled target data?
• RQ5 Can the proposed self-supervised approach complement pixel-level domain adaptation methods?

Key findings

• Achieves state-of-the-art results on four of seven standard object-recognition domain-adaptation benchmarks and competitive performance on segmentation adaptation.
• Demonstrates that joint self-supervision on both domains can align representations and enable the source-trained classifier to generalize to the target.
• Shows that combining multiple self-supervised tasks yields better adaptation than using a single task.
• Finds that the method can be effectively composed with pixel-level adaptation techniques for further gains (e.g., CyCADA baseline improvements).
• Reports that the approach can handle segmentation adaptation (GTA5 to Cityscapes) with significant improvements over source-only baselines and gains when combined with CyCADA.