[Paper Review] Unsupervised Domain Generalization by Learning a Bridge Across Domains

Sivan Harary, Eli Schwartz+12 more|arXiv (Cornell University)|Dec 4, 2021

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes Unsupervised Domain Generalization (UDG) via a learnable Bridge Across Domains (BrAD), an auxiliary edge-regularized visual domain that enables self-supervised contrastive learning to align features across diverse, unlabeled source and target domains. By jointly training image-to-BrAD mappings and a contrastive representation model, the method achieves state-of-the-art performance—up to 14% accuracy gain in UDG and 13.3% in FUDA—without any source domain labels, enabling generalization to unseen domains and classes.

ABSTRACT

The ability to generalize learned representations across significantly different visual domains, such as between real photos, clipart, paintings, and sketches, is a fundamental capacity of the human visual system. In this paper, different from most cross-domain works that utilize some (or full) source domain supervision, we approach a relatively new and very practical Unsupervised Domain Generalization (UDG) setup of having no training supervision in neither source nor target domains. Our approach is based on self-supervised learning of a Bridge Across Domains (BrAD) - an auxiliary bridge domain accompanied by a set of semantics preserving visual (image-to-image) mappings to BrAD from each of the training domains. The BrAD and mappings to it are learned jointly (end-to-end) with a contrastive self-supervised representation model that semantically aligns each of the domains to its BrAD-projection, and hence implicitly drives all the domains (seen or unseen) to semantically align to each other. In this work, we show how using an edge-regularized BrAD our approach achieves significant gains across multiple benchmarks and a range of tasks, including UDG, Few-shot UDA, and unsupervised generalization across multi-domain datasets (including generalization to unseen domains and classes).

Motivation & Objective

• To address the challenge of domain generalization in real-world scenarios where no labeled data is available in source or target domains.
• To enable generalization to unseen domains and novel classes without additional fine-tuning or supervision.
• To develop a self-supervised framework that implicitly aligns representations across diverse visual domains using a learnable bridge domain.
• To reduce reliance on pre-trained models or domain-specific supervision while maintaining robustness across domain shifts.

Proposed method

• Introduces a learnable Bridge Across Domains (BrAD) as an auxiliary visual domain, specifically designed to be easily mapable from all source domains via image-to-image transformations.
• Employs edge-regularized BrAD, where the bridge domain is learned to preserve semantic structure while suppressing noise, using a HED-based network or learned edge heads.
• Jointly trains a contrastive self-supervised representation model with domain-specific mappings to BrAD, enforcing feature-level semantic alignment across domains.
• Applies a contrastive loss (L_contrast) that aligns features of real images with their BrAD projections, and a regularization loss (L_Ω) that encourages consistency in edge maps across domains.
• Uses a dual-branch encoder with shared backbone weights to ensure domain-invariant representations, with the BrAD serving as a shared semantic anchor.
• Enables inference without BrAD mappings by relying on the shared, aligned feature space, allowing zero-shot transfer to unseen domains and classes.

Experimental results

Research questions

• RQ1 Can a self-supervised model achieve strong domain generalization without any labeled data in source or target domains?
• RQ2 Can a learnable, edge-regularized bridge domain effectively align representations across diverse visual domains such as real images, sketches, and paintings?
• RQ3 Does the proposed BrAD framework generalize to unseen domains and novel classes without fine-tuning?
• RQ4 How does the choice of BrAD (e.g., Canny, HED, learned) affect performance in UDG and FUDA tasks?
• RQ5 To what extent is the method robust to the absence of pre-trained models, such as BSDS500 weights for HED?

Key findings

• The proposed method achieves a 14% accuracy gain over the previous SOTA in Unsupervised Domain Generalization (UDG) on the DomainNet benchmark.
• On the FUDA benchmark, the method improves performance by 13.3% over the previous SOTA, demonstrating strong few-shot generalization capability.
• Even without any BSDS500 pretraining, the method maintains near-identical performance (±1.2% change), proving robustness to pretraining dependencies.
• The learned BrAD outperforms both Canny and HED edge maps by preserving fine internal details while suppressing noise, as shown in qualitative comparisons.
• The method generalizes effectively to unseen domains and novel classes without additional fine-tuning, as validated across multiple multi-domain benchmarks.
• The ablation study confirms that all components—BrAD learning, edge regularization, and contrastive alignment—contribute significantly to performance gains.