[Paper Review] Deeper, Broader and Artier Domain Generalization

Da Li, Yongxin Yang+2 more|arXiv (Cornell University)|Oct 9, 2017

Domain Adaptation and Few-Shot Learning28 references4 citations

TL;DR

This paper proposes a low-rank parameterized CNN for end-to-end domain generalization that outperforms prior methods on a new, more challenging benchmark dataset, PACS, which spans photo, art painting, cartoon, and sketch domains. The method uses 2-hot encoded domain descriptors and low-rank factorization to share knowledge across domains while reducing overfitting, achieving state-of-the-art performance on the PACS benchmark with a 69.21% average accuracy.

ABSTRACT

The problem of domain generalization is to learn from multiple training domains, and extract a domain-agnostic model that can then be applied to an unseen domain. Domain generalization (DG) has a clear motivation in contexts where there are target domains with distinct characteristics, yet sparse data for training. For example recognition in sketch images, which are distinctly more abstract and rarer than photos. Nevertheless, DG methods have primarily been evaluated on photo-only benchmarks focusing on alleviating the dataset bias where both problems of domain distinctiveness and data sparsity can be minimal. We argue that these benchmarks are overly straightforward, and show that simple deep learning baselines perform surprisingly well on them. In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning. Secondly, we develop a DG benchmark dataset covering photo, sketch, cartoon and painting domains. This is both more practically relevant, and harder (bigger domain shift) than existing benchmarks. The results show that our method outperforms existing DG alternatives, and our dataset provides a more significant DG challenge to drive future research.

Motivation & Objective

• To address the limitations of existing domain generalization (DG) benchmarks, which are overly simplistic and based on photo-only domains with minimal domain shift.
• To develop a more practical and challenging DG benchmark that reflects real-world scenarios where target domains (e.g., sketches, paintings) are data-scarce and visually distinct from photos.
• To propose a novel deep learning method that enables end-to-end learning of domain-agnostic features and classifiers through low-rank parameterized weight generation.
• To demonstrate that simple deep learning baselines can outperform purpose-designed DG methods when evaluated on more realistic, high-shift benchmarks.

Proposed method

• Proposes a low-rank parameterized convolutional neural network (CNN) that reduces model parameters and enables flexible, learnable domain sharing across layers.
• Uses 2-hot encoding of domain descriptors to generate shared, domain-agnostic classifier weights via a tensor-based weight generator network.
• Applies low-rank decomposition to the weight generator parameters, allowing dynamic and layer-specific control over the degree of knowledge sharing between domains.
• Performs Tucker decomposition on the tuned model to analyze learned layer-wise sharing strength, revealing that early layers share more knowledge than later fully connected layers.
• Employs end-to-end training on multiple source domains (photo, painting, cartoon, sketch) to jointly optimize feature extraction and domain-agnostic classification.
• Introduces a novel ablation framework to evaluate the contribution of each component: domain descriptor encoding, low-rank factorization, and multi-layer application.

Experimental results

Research questions

• RQ1 Can a simple end-to-end deep learning baseline outperform purpose-designed domain generalization methods when evaluated on a more realistic and challenging benchmark?
• RQ2 How does the degree of domain shift in a new benchmark (PACS) compare to existing photo-only benchmarks in terms of difficulty and practical relevance?
• RQ3 To what extent do low-rank parameterization and 2-hot encoded domain descriptors improve generalization performance across diverse visual domains?
• RQ4 How does the learned sharing strength vary across different layers of a CNN in a domain generalization setting?
• RQ5 What is the contribution of multi-layer low-rank weight generation versus single-layer fine-tuning in domain generalization?

Key findings

• The proposed method, Ours-Full, achieves a 69.21% average accuracy on the PACS benchmark, outperforming all prior DG methods, including uDICA and D-MTAE-1HNN.
• The ablation study shows that each component—2-hot encoding, low-rank decomposition, and multi-layer application—contributes incrementally to performance, with Ours-Full surpassing 2HE+Decomp-Last by 5.11% on average.
• The model learns flexible, layer-specific sharing strength: early convolutional layers (Conv1–Conv3) show high sharing, while later layers (FC6–FC8) show low sharing, with intermediate layers varying based on the unseen domain.
• When sketch is the unseen domain, the model learns higher sharing among photo, painting, and cartoon domains, reflecting their relative visual similarity to sketch.
• The Tucker decomposition reveals that the model automatically learns that sketch is the most dissimilar domain, resulting in lower sharing strength when it is excluded from the training domains.
• The method outperforms strong baselines like Deep-ALL and SVM, demonstrating that end-to-end deep learning with explicit DG inductive bias is more effective than shallow or hand-designed approaches.