[Paper Review] DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation

Bharath Bhushan Damodaran, Benjamin Kellenberger+3 more|arXiv (Cornell University)|Mar 27, 2018

Domain Adaptation and Few-Shot LearningComputer Science51 references372 citations

TL;DR

DeepJDOT learns a shared latent space and a target-domain classifier by jointly optimizing deep representations and a joint distribution optimal transport coupling between source and target, achieving strong unsupervised domain adaptation performance.

ABSTRACT

In computer vision, one is often confronted with problems of domain shifts, which occur when one applies a classifier trained on a source dataset to target data sharing similar characteristics (e.g. same classes), but also different latent data structures (e.g. different acquisition conditions). In such a situation, the model will perform poorly on the new data, since the classifier is specialized to recognize visual cues specific to the source domain. In this work we explore a solution, named DeepJDOT, to tackle this problem: through a measure of discrepancy on joint deep representations/labels based on optimal transport, we not only learn new data representations aligned between the source and target domain, but also simultaneously preserve the discriminative information used by the classifier. We applied DeepJDOT to a series of visual recognition tasks, where it compares favorably against state-of-the-art deep domain adaptation methods.

Motivation & Objective

• Motivate robust unsupervised domain adaptation when target labels are unavailable

Proposed method

• Define an embedding g and a classifier f within a CNN to map inputs to a semantic latent space
• Formulate a joint distribution OT objective over embedded representations and labels, minimizing a cost that couples source and target samples
• Solve the OT coupling via stochastic mini-batches for scalability and integrate it into standard backpropagation
• Update the embedding g and classifier f jointly to minimize both source supervision and the OT-based alignment
• Optionally include the source-domain loss to prevent catastrophic forgetting and maintain discrimination across domains

Experimental results

Research questions

• RQ1 How can joint distribution optimal transport be integrated into deep models to align source and target domains semantically?
• RQ2 Does learning the coupling in deep feature spaces improve target-domain discrimination compared to aligning in input space?
• RQ3 Can a stochastic, mini-batch OT approach scale to large datasets while preserving alignment quality?
• RQ4 Does incorporating source-domain supervision alongside the OT objective prevent catastrophic forgetting during adaptation?

Key findings

• DeepJDOT outperforms several state-of-the-art unsupervised domain adaptation methods on digits recognition tasks and Office-Home dataset.
• The method learns embeddings that align source and target distributions and maximize class separation in a shared latent space.
• A stochastic optimization procedure enables scalable training by solving small OT couplings within mini-batches.
• Ablation studies show that combining the joint embedding/label alignment with source supervision yields better results than using components in isolation.
• Visualizations indicate DeepJDOT achieves near-perfect alignment of source and target samples in the embedding space and clear class separation.