[Paper Review] Towards Principled Disentanglement for Domain Generalization

Hanlin Zhang, Yifan Zhang+4 more|arXiv (Cornell University)|Nov 27, 2021

Domain Adaptation and Few-Shot Learning66 references4 citations

TL;DR

This paper proposes DDG, a principled disentanglement framework for domain generalization that formulates invariant representation learning as a constrained optimization problem, jointly training semantic and variation encoders via primal-dual optimization. DDG achieves state-of-the-art OOD generalization performance by disentangling intrinsic semantics from nuisance variations without requiring domain labels, while enabling domain-agnostic data augmentation through learned disentangled factors.

ABSTRACT

A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data, in part due to spurious correlations. To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG). We relax this non-trivial constrained optimization problem to a tractable form with finite-dimensional parameterization and empirical approximation. Then a theoretical analysis of the extent to which the above transformations deviates from the original problem is provided. Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization. In contrast to traditional approaches based on domain adversarial training and domain labels, DDG jointly learns semantic and variation encoders for disentanglement, enabling flexible manipulation and augmentation on training data. DDG aims to learn intrinsic representations of semantic concepts that are invariant to nuisance factors and generalizable across domains. Comprehensive experiments on popular benchmarks show that DDG can achieve competitive OOD performance and uncover interpretable salient structures within data.

Motivation & Objective

• To address poor out-of-distribution (OOD) generalization in deep learning caused by spurious correlations and domain shift.
• To formalize domain generalization as a constrained optimization problem based on disentanglement of semantics and variations.
• To develop a method that learns invariant representations without relying on expensive or impractical domain labels.
• To provide theoretical guarantees on the approximation quality of the proposed optimization transformation.
• To enable flexible, domain-agnostic data augmentation through disentangled representation learning.

Proposed method

• Formalize domain generalization as a disentanglement-constrained optimization problem (DDG), where semantics and variations are explicitly separated.
• Relax the non-trivial constrained problem into a tractable form using finite-dimensional parameterization and empirical approximation.
• Propose a primal-dual algorithm that jointly optimizes semantic and variation encoders to enforce disentanglement and invariance.
• Parameterize the semantic and variation encoders to model both intra- and inter-domain variations, enabling flexible data manipulation.
• Use the learned variation encoder to generate domain-agnostic data augmentations by composing semantic and variation factors.
• Provide theoretical analysis of the parameterization gap and empirical duality gap to bound approximation error.

Experimental results

Research questions

• RQ1 Can disentanglement of semantics and variations be formally framed as a constrained optimization problem to improve OOD generalization?
• RQ2 How well does the relaxed optimization formulation approximate the original constrained problem in practice?
• RQ3 Can a joint learning framework of semantic and variation encoders outperform domain-adversarial or domain-label-dependent methods in DG?
• RQ4 To what extent can the disentangled representation enable effective, domain-agnostic data augmentation?
• RQ5 Does the proposed method achieve consistent performance gains across diverse benchmarks under a standardized evaluation protocol?

Key findings

• DDG achieves state-of-the-art performance on PACS, outperforming MBDG by 3.1% on average (88.9% vs. 85.8%)
• On the PACS test domain S, DDG achieves 84.3% accuracy, slightly below MBDG’s 85.0%, but still competitive across all domains
• DDG consistently outperforms MBDG and its variant (MBDG Reg) on all domains except S, demonstrating robustness to domain shift
• The method enables effective, domain-agnostic data augmentation through learned disentangled factors, without requiring gradient estimation or domain-specific knowledge
• Qualitative results show that DDG successfully disentangles both intra-domain (e.g., thickness) and inter-domain (e.g., rotated angles) variations
• The model generates realistic, diverse samples via interpolation and factor swapping, confirming strong disentanglement and generalization capability