[Paper Review] Domain2Vec: Deep Domain Generalization

Aniket Anand Deshmukh, Ankit Bansal+1 more|arXiv (Cornell University)|Jul 9, 2018

Domain Adaptation and Few-Shot Learning17 references4 citations

TL;DR

Domain2Vec (D2V) proposes a deep neural network architecture that learns domain-specific embeddings by modeling statistical features of data distributions, then combines these embeddings with input features to improve zero-shot domain generalization. It outperforms state-of-the-art methods on image classification benchmarks like VLCS, achieving 70.82% average accuracy in leave-one-domain-out evaluation.

ABSTRACT

We address the problem of domain generalization where a decision function is learned from the data of several related domains, and the goal is to apply it on an unseen domain successfully. It is assumed that there is plenty of labeled data available in source domains (also called as training domain), but no labeled data is available for the unseen domain (also called a target domain or test domain). We propose a novel neural network architecture, Domain2Vec (D2V) that learns domain-specific embedding and then uses this embedding to generalize the learning across related domains. The proposed algorithm, D2V extends the idea of distribution regression and kernelized domain generalization to the neural networks setting. We propose a neural network architecture to learn domain-specific embedding and then use this embedding along with the data point specific features to label it. We show the effectiveness of the architecture by accurately estimating domain to domain similarity. We evaluate our algorithm against standard domain generalization datasets for image classification and outperform other state of the art algorithms.

Motivation & Objective

• Address the challenge of domain generalization where models must classify data from an unseen domain with no labeled training data.
• Learn robust, domain-specific representations that capture underlying distributional differences across multiple source domains.
• Improve generalization performance by combining data-point features with learned domain embeddings in a unified neural network framework.
• Demonstrate that the learned domain embeddings accurately reflect true domain similarity, enabling better transfer to unseen domains.
• Extend the idea of distribution regression and kernelized domain generalization to a deep neural network setting with end-to-end training.

Proposed method

• Propose a dual-stream neural network architecture: a main network learns domain-specific embeddings from marginal data distributions, while a task network learns classifier weights for each domain.
• Use statistical moments or empirical characteristic functions of input data distributions as domain embeddings, which are then concatenated with input features.
• Train the model end-to-end to minimize expected classification loss over the extended feature space $\mathcal{X} \times \mathcal{P}_{\mathcal{X}}$, where $\mathcal{P}_{\mathcal{X}}$ represents the empirical distribution of each domain.
• Leverage a shared representation between the main and task networks to enable knowledge transfer and improve generalization across domains.
• Estimate domain similarity using the RBF kernel on learned embeddings: $\exp(-\|d_p - d_q\|^2 / \sigma^2)$, which correlates with true angular similarity in synthetic experiments.
• Apply random search to optimize hyperparameters including learning rate, number of hidden units, and weight decay for both networks.

Experimental results

Research questions

• RQ1 Can a deep neural network learn meaningful domain-specific embeddings that capture distributional differences across multiple source domains?
• RQ2 Does combining learned domain embeddings with input features improve zero-shot generalization to an unseen target domain?
• RQ3 How well do the learned domain embeddings reflect true domain similarity compared to ground-truth distributional differences?
• RQ4 Can the proposed architecture outperform existing state-of-the-art methods in domain generalization benchmarks?
• RQ5 Is the model robust to variations in the number of training domains and examples per domain?

Key findings

• On a 2D synthetic dataset, Domain2Vec improved classification accuracy from 73% (pooling NN) to 96% by learning domain embeddings that reflect true domain similarity.
• The learned domain embeddings showed strong correlation with ground-truth domain similarity (based on rotation angles), as demonstrated by heatmap comparisons.
• On the VLCS benchmark using leave-one-domain-out evaluation, Domain2Vec achieved 70.82% average accuracy, surpassing all baselines including L-SVM (59.63%), 1 HNN (65.46%), and D-MTAE (69.65%).
• Performance improved with more training domains and more examples per domain, indicating scalability and data efficiency.
• The model achieved 92.50% accuracy on the Caltech-101 target domain when trained on VOC2007, LabelMe, and SUN09, outperforming the next best method (D-MTAE at 92.43%).
• The architecture demonstrated consistent gains across all four target domains in VLCS, with the highest improvement observed on the SUN09 target (62.02% vs. 61.85% for D-MTAE).