[Paper Review] Domain Separation Networks

Konstantinos Bousmalis, George Trigeorgis+3 more|arXiv (Cornell University)|Aug 22, 2016

Domain Adaptation and Few-Shot Learning31 references588 citations

TL;DR

DSN explicitly models private (domain-specific) and shared representations for unsupervised domain adaptation, combining reconstruction and orthogonality losses with similarity guidance to outperform prior methods.

ABSTRACT

The cost of large scale data collection and annotation often makes the application of machine learning algorithms to new tasks or datasets prohibitively expensive. One approach circumventing this cost is training models on synthetic data where annotations are provided automatically. Despite their appeal, such models often fail to generalize from synthetic to real images, necessitating domain adaptation algorithms to manipulate these models before they can be successfully applied. Existing approaches focus either on mapping representations from one domain to the other, or on learning to extract features that are invariant to the domain from which they were extracted. However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain. We suggest that explicitly modeling what is unique to each domain can improve a model's ability to extract domain-invariant features. Inspired by work on private-shared component analysis, we explicitly learn to extract image representations that are partitioned into two subspaces: one component which is private to each domain and one which is shared across domains. Our model is trained not only to perform the task we care about in the source domain, but also to use the partitioned representation to reconstruct the images from both domains. Our novel architecture results in a model that outperforms the state-of-the-art on a range of unsupervised domain adaptation scenarios and additionally produces visualizations of the private and shared representations enabling interpretation of the domain adaptation process.

Motivation & Objective

• Motivate reducing annotation costs by leveraging synthetic data and domain adaptation.
• Propose a two-subspace representation: private (domain-specific) and shared (domain-invariant).
• Train with task loss on source domain plus reconstruction and orthogonality losses to separate representations.
• Enable visualization/interpretability of private and shared components during adaptation.

Proposed method

• Introduce shared encoder Ec and private encoder Ep per domain.
• Use a shared decoder D to reconstruct inputs from Ec and Ep representations.
• Impose a difference loss to encourage orthogonality between private and shared components (L_difference).
• Impose a similarity loss to align shared representations across domains (L_similarity) using either DANN (with GRL) or MMD.
• Train with task loss on source labels and reconstruction loss on both domains; apply domain adaptation losses after a warm-up period.
• Share architecture and training routine across multiple domain adaptation scenarios.

Experimental results

Research questions

• RQ1 How can explicitly modeling domain-private and domain-shared representations improve unsupervised domain adaptation?
• RQ2 Do private-shared partitioning and reconstruction losses yield better cross-domain generalization than prior mapping or shared-representation methods?
• RQ3 Which similarity objective (DANN vs. MMD) best promotes cross-domain alignment within DSN?
• RQ4 Can DSN provide interpretable visualizations of private vs shared representations during adaptation.

Key findings

• DSN with DANN similarity loss outperforms CORAL, MMD, and standard DANN baselines across multiple unsupervised domain adaptation tasks.
• In ablations, removing the difference (orthogonality) loss degrades performance across tasks.
• Replacing scale-invariant reconstruction with standard L2 MSE reduces performance, validating the proposed reconstruction loss.
• DSN achieves strong accuracy on MNIST→MNIST-M, Synth Digits→SVHN, SVHN→MNIST, and Synth Signs→GTSRB tasks.
• DSN also yields competitive pose estimation results in the Synth Objects→LINEMOD scenario.