[Paper Review] Learning to Diversify for Single Domain Generalization

Zijian Wang, Yadan Luo+3 more|arXiv (Cornell University)|Aug 26, 2021

Domain Adaptation and Few-Shot Learning51 references4 citations

TL;DR

This paper proposes Learning-to-Diversify (L2D), a novel method for single domain generalization (Single-DG) that uses a style-complement module to generate diverse, out-of-distribution images from a single source domain. By iteratively minimizing mutual information between generated and source samples (for diversity) and maximizing it among same-class samples (for invariance), the model achieves state-of-the-art performance, surpassing prior methods by up to 25.14% on benchmark datasets.

ABSTRACT

Domain generalization (DG) aims to generalize a model trained on multiple source (i.e., training) domains to a distributionally different target (i.e., test) domain. In contrast to the conventional DG that strictly requires the availability of multiple source domains, this paper considers a more realistic yet challenging scenario, namely Single Domain Generalization (Single-DG), where only one source domain is available for training. In this scenario, the limited diversity may jeopardize the model generalization on unseen target domains. To tackle this problem, we propose a style-complement module to enhance the generalization power of the model by synthesizing images from diverse distributions that are complementary to the source ones. More specifically, we adopt a tractable upper bound of mutual information (MI) between the generated and source samples and perform a two-step optimization iteratively: (1) by minimizing the MI upper bound approximation for each sample pair, the generated images are forced to be diversified from the source samples; (2) subsequently, we maximize the MI between the samples from the same semantic category, which assists the network to learn discriminative features from diverse-styled images. Extensive experiments on three benchmark datasets demonstrate the superiority of our approach, which surpasses the state-of-the-art single-DG methods by up to 25.14%.

Motivation & Objective

• To address the challenge of domain generalization when only one source domain is available during training, a scenario known as Single-DG.
• To overcome the limited data diversity in Single-DG that hampers model generalization to unseen target domains.
• To improve model robustness by generating diverse, out-of-distribution images that challenge the classifier during training.
• To learn style-invariant representations through mutual information maximization among same-class samples.
• To develop a min-max optimization framework that alternately enhances sample diversity and feature discriminability.

Proposed method

• A style-complement module is introduced to synthesize images with novel, diverse styles not present in the original source domain.
• The method uses a tractable upper bound of mutual information (MI) to enforce diversity between generated and source samples in the latent space.
• In a two-step iterative optimization, the model first minimizes MI between generated and source sample pairs to maximize distributional divergence.
• Subsequently, the model maximizes MI between samples of the same semantic category to learn discriminative, style-invariant features.
• The style-complement module and the main task model engage in a min-max game, where generated images act as adversarial challenges to improve generalization.
• The framework is trained end-to-end with a loss function combining diversity (via MI minimization) and invariance (via MI maximization) objectives.

Experimental results

Research questions

• RQ1 Can a single-domain generalization model achieve strong generalization performance when only one source domain is available during training?
• RQ2 How can effective data augmentation be achieved in the absence of multiple source domains to simulate domain shift?
• RQ3 To what extent can mutual information optimization enhance feature robustness and model generalization in Single-DG?
• RQ4 Does generating diverse, out-of-distribution images improve the model's ability to generalize to unseen target domains?
• RQ5 How does the interplay between diversity and invariance learning affect performance in the absence of domain identifiers?

Key findings

• The proposed L2D method achieves state-of-the-art performance on three benchmark datasets—digits recognition, corrupted CIFAR-10, and PACS—surpassing existing Single-DG methods by up to 25.14% in accuracy.
• Ablation studies show that removing the style-complement module leads to a significant performance drop, confirming its critical role in enhancing generalization.
• The removal of style modification or mutual information optimization components results in performance degradation of 1.35% and 3.76% respectively, demonstrating the importance of both components.
• Sensitivity analysis indicates that using k=6 transformations achieves optimal diversity, with performance stabilizing beyond k=5 and peaking at α₁=α₂=1.
• t-SNE visualizations confirm that L2D produces tighter, better-separated class clusters compared to baselines, indicating improved feature discriminability and domain invariance.
• The method generalizes well to standard domain generalization under the leave-one-domain-out protocol, validating its broader applicability beyond Single-DG.