[Paper Review] Dynamic Domain Generalization

Zhishu Sun, Z. Shen+5 more|arXiv (Cornell University)|May 27, 2022

Domain Adaptation and Few-Shot Learning5 citations

TL;DR

This paper proposes Dynamic Domain Generalization (DDG), a novel training-free framework that enables deep neural networks to adapt to agnostic target domains by dynamically adjusting network parameters via a meta-adjuster during inference. By leveraging DomainMix to simulate diverse domains and asymmetric kernel templates to generate adaptive parameters, DDG achieves state-of-the-art performance on PACS, Office-Home, and DomainNet benchmarks without any fine-tuning on target data.

ABSTRACT

Domain generalization (DG) is a fundamental yet very challenging research topic in machine learning. The existing arts mainly focus on learning domain-invariant features with limited source domains in a static model. Unfortunately, there is a lack of training-free mechanism to adjust the model when generalized to the agnostic target domains. To tackle this problem, we develop a brand-new DG variant, namely Dynamic Domain Generalization (DDG), in which the model learns to twist the network parameters to adapt the data from different domains. Specifically, we leverage a meta-adjuster to twist the network parameters based on the static model with respect to different data from different domains. In this way, the static model is optimized to learn domain-shared features, while the meta-adjuster is designed to learn domain-specific features. To enable this process, DomainMix is exploited to simulate data from diverse domains during teaching the meta-adjuster to adapt to the upcoming agnostic target domains. This learning mechanism urges the model to generalize to different agnostic target domains via adjusting the model without training. Extensive experiments demonstrate the effectiveness of our proposed method. Code is available at: https://github.com/MetaVisionLab/DDG

Motivation & Objective

• To address the limitation of static domain generalization models that cannot adapt to agnostic target domains without fine-tuning.
• To develop a training-free mechanism that enables dynamic adaptation of model parameters during inference for improved generalization to unseen domains.
• To disentangle domain-shared and domain-specific features through a decoupled static-dynamic parameter structure.
• To train the meta-adjuster effectively using simulated novel domains, avoiding reliance on domain labels or real target data.

Proposed method

• Decompose network parameters into a static part (learned via standard training) and a dynamic part (modulated by the meta-adjuster at inference time).
• Design a meta-adjuster that generates dynamic coefficients per input instance using instance embeddings as input.
• Use asymmetric kernel templates to form dynamic parameters via a low-dimensional linear combination, enabling diverse and adaptive modulation.
• Employ DomainMix to simulate data from novel, unseen domains by convex interpolation across source domains, providing diverse training signals for the meta-adjuster.
• Train the meta-adjuster to minimize the loss on simulated domains, enabling it to generalize to real agnostic target domains without fine-tuning.
• Apply t-SNE visualization and kernel magnitude analysis to validate the domain-aware behavior of dynamic coefficients and enhanced kernel structures.

Experimental results

Research questions

• RQ1 Can a training-free mechanism dynamically adapt a deep model to unseen target domains without fine-tuning?
• RQ2 How can a meta-adjuster learn to generate domain-specific parameter adjustments without explicit domain labels?
• RQ3 To what extent does DomainMix simulation of novel domains improve the meta-adjuster's ability to generalize to real agnostic targets?
• RQ4 How do asymmetric kernel templates enhance the expressiveness and diversity of dynamic parameter generation?
• RQ5 Does the dynamic adaptation mechanism lead to better domain-invariant feature learning compared to static domain generalization?

Key findings

• DDG achieves state-of-the-art performance on PACS, Office-Home, and DomainNet benchmarks, outperforming existing domain generalization methods without any fine-tuning on target domains.
• Ablation studies show that removing DomainMix leads to significant performance degradation, confirming its critical role in simulating diverse domains for meta-adjuster training.
• t-SNE visualization confirms that dynamic coefficients are domain-aware and separate based on domain distribution, even without explicit domain supervision.
• Kernel magnitude visualization shows that asymmetric kernel templates strengthen the central 'skeleton' of convolutional kernels, enhancing feature learning capacity.
• The dynamic network learns more robust and generalizable features than the static ResNet baseline, as evidenced by improved t-SNE clustering of features across domains.
• The method successfully generalizes to real agnostic target domains by learning to adapt via parameter twisting at inference, demonstrating the feasibility of training-free domain adaptation.