[Paper Review] Domain Generalisation via Domain Adaptation: An Adversarial Fourier Amplitude Approach

Minyoung Kim, Da Li+1 more|arXiv (Cornell University)|Feb 23, 2023

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes AGFA, a novel domain generalization method that formulates domain generalization as adversarial domain adaptation by synthesizing worst-case target domains using Fourier amplitude manipulation. By combining source domain phases with adversarially trained, style-altering Fourier amplitudes and leveraging Bayesian maximum classifier discrepancy, AGFA achieves state-of-the-art performance on DomainBed and PACS benchmarks, outperforming prior methods by up to 10% in single-source generalization.

ABSTRACT

We tackle the domain generalisation (DG) problem by posing it as a domain adaptation (DA) task where we adversarially synthesise the worst-case target domain and adapt a model to that worst-case domain, thereby improving the model's robustness. To synthesise data that is challenging yet semantics-preserving, we generate Fourier amplitude images and combine them with source domain phase images, exploiting the widely believed conjecture from signal processing that amplitude spectra mainly determines image style, while phase data mainly captures image semantics. To synthesise a worst-case domain for adaptation, we train the classifier and the amplitude generator adversarially. Specifically, we exploit the maximum classifier discrepancy (MCD) principle from DA that relates the target domain performance to the discrepancy of classifiers in the model hypothesis space. By Bayesian hypothesis modeling, we express the model hypothesis space effectively as a posterior distribution over classifiers given the source domains, making adversarial MCD minimisation feasible. On the DomainBed benchmark including the large-scale DomainNet dataset, the proposed approach yields significantly improved domain generalisation performance over the state-of-the-art.

Motivation & Objective

• To address domain generalization (DG) in the absence of target domain data, where models often fail due to distribution shift.
• To improve model robustness by simulating the most challenging yet semantically valid target domain for training.
• To unify data augmentation and domain alignment through a principled, adversarial framework grounded in domain adaptation theory.
• To leverage Fourier domain decomposition—separating amplitude (style) and phase (semantics)—for semantic-preserving data synthesis.
• To enable effective adversarial training of worst-case target data without compromising semantic content.

Proposed method

• The method formulates domain generalization as adversarial domain adaptation by generating a worst-case target domain using Fourier amplitude synthesis.
• It leverages the conjecture that Fourier amplitude controls image style while phase preserves semantics, enabling style transfer without semantic alteration.
• A variational Bayesian approach models the hypothesis space of classifiers as a posterior distribution, enabling efficient maximum classifier discrepancy (MCD) minimization.
• An adversarial generator is trained to produce Fourier amplitude images that maximize MCD, simulating the most challenging target domain.
• The classifier is jointly trained to minimize MCD on the synthesized worst-case domain, improving robustness.
• The framework combines adversarial training of the amplitude generator with Bayesian modeling of classifier uncertainty to ensure stable and generalizable adaptation.

Figure 1: Overall training flow of the proposed approach (AGFA). We generate target-domain data by synthesizing Fourier amplitude images trained adversarially. See main text in Sec. 3 for details.

Experimental results

Research questions

• RQ1 Can we effectively simulate a worst-case target domain for domain generalization without access to real target data?
• RQ2 Can Fourier amplitude-based data augmentation preserve semantic content while introducing challenging style variations?
• RQ3 Can Bayesian modeling of classifier hypotheses enable effective maximum classifier discrepancy minimization in the DG setting?
• RQ4 Does adversarial synthesis of worst-case domains lead to improved generalization performance across diverse benchmarks?
• RQ5 How does the proposed method compare to state-of-the-art methods in both standard and single-source domain generalization settings?

Key findings

• On the DomainBed benchmark with the ResNet-18 backbone, AGFA achieved an average accuracy of 73.8%, surpassing the previous SOTA method (MMD) by 0.5%.
• On the large-scale DomainNet dataset, AGFA achieved state-of-the-art performance, demonstrating robustness across diverse domain shifts.
• In the single-source generalization setting on PACS with ResNet-18, AGFA outperformed the best prior method (EFDMix) by 10.2 percentage points in average accuracy.
• With ResNet-50, AGFA achieved an average accuracy of 68.7%, exceeding EFDMix by 7.5 percentage points.
• Visual analysis confirmed that Fourier-based generated images retain salient semantic edges, unlike pixel-based generators that produce random noise.
• The ablation study showed that the combination of Bayesian MCD and Fourier amplitude synthesis was essential for performance, with each component contributing significantly to robustness.

Figure 2: Illustration of the SMCD loss on three different hypothesis spaces $\mathcal{H}_{|S}$ shown in three panels. For $C=3$ -way classification case, each panel shows the class logit scores (Gaussian random) $f_{j}(x)\sim\mathcal{N}(\mu_{j}(x),\sigma_{j}(x)^{2})$ for $j=1,2,3$ , at some input $