[论文解读] Generalizing to Unseen Domains: A Survey on Domain Generalization

Jindong Wang, Cuiling Lan+7 more|arXiv (Cornell University)|Mar 2, 2021

Domain Adaptation and Few-Shot Learning被引用 96

一句话总结

对领域泛化（DG）的全面综述：形式定义、理论、算法分类、数据集、应用以及开放研究方向，包括开源代码库 DeepDG。

ABSTRACT

Machine learning systems generally assume that the training and testing distributions are the same. To this end, a key requirement is to develop models that can generalize to unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. First, we provide a formal definition of domain generalization and discuss several related fields. We then thoroughly review the theories related to domain generalization and carefully analyze the theory behind generalization. We categorize recent algorithms into three classes: data manipulation, representation learning, and learning strategy, and present several popular algorithms in detail for each category. Third, we introduce the commonly used datasets, applications, and our open-sourced codebase for fair evaluation. Finally, we summarize existing literature and present some potential research topics for the future.

研究动机与目标

• 定义领域泛化并将其与相关领域区分开。
• 综述与 DG 相关的理论基础与泛化界限。
• 按数据处理、表示学习和学习策略对主要 DG 算法进行分类和分析。
• 讨论常用数据集、应用，并提供开源评估资源 DeepDG。
• 概述 DG 的未来研究方向及潜在挑战。

提出的方法

• 正式将 DG 表述为具有训练源域和未见目标域的情形。
• 回顾并将 DG 理论与领域自适应通过如 HΔH-divergence 的界限以及协变量漂移分析联系起来。
• 提供 DG 方法的分类法：数据处理、表示学习和学习策略，并给出子类别。
• 讨论领域不变表示、特征解耦和因果启发的方法。
• 介绍 DeepDG，一个用于 DG 评估与实验的开源代码库。

Figure 1: Examples from the dataset PACS [ 2 ] for domain generalization. The training set is composed of images belonging to domains of sketch, cartoon, and art paintings. DG aims to learn a generalized model that performs well on the unseen target domain of photos.

实验结果

研究问题

• RQ1 哪些正式定义及与相关领域（例如领域自适应、迁移学习）之间的关系可以刻画 DG？
• RQ2 哪些理论界限解释 DG 泛化，以及在实际中如何估计？
• RQ3 DG 方法如何分类，每个类别中的代表性算法有哪些？
• RQ4 用于 DG 的数据集与评估协议有哪些，如何促进公平评估？
• RQ5 DG 的未来方向和开放挑战有哪些？

主要发现

• DG 被形式化为从多个带标签的源域学习，以泛化到分布不同的未见目标域。
• 理论 DG 界限利用如 HΔH-divergence 和协变量漂移等概念，指导领域不变表示。
• DG 方法分为数据处理、表示学习和学习策略，每个类别都有多种有影响力的技术。
• 领域不变表示和特征解耦是实现 DG 的核心主题，包含多种核方法、对抗方法和正则化方法。
• 提供开源 DeepDG 代码库，以支持在 DG 基准和数据集上的公平评估。

Figure 2: Illustration of domain generalization. Adapted from [ 6 ] .