[论文解读] Transfer Adaptation Learning: A Decade Survey

Lei Zhang, Xinbo Gao|arXiv (Cornell University)|Mar 12, 2019

Domain Adaptation and Few-Shot Learning被引用 65

一句话总结

本综述在迁移学习和领域自适应下统一为 Transfer Adaptation Learning (TAL)，并回顾过去十年的五大适应机制、挑战与未来方向。

ABSTRACT

The world we see is ever-changing and it always changes with people, things, and the environment. Domain is referred to as the state of the world at a certain moment. A research problem is characterized as transfer adaptation learning (TAL) when it needs knowledge correspondence between different moments/domains. Conventional machine learning aims to find a model with the minimum expected risk on test data by minimizing the regularized empirical risk on the training data, which, however, supposes that the training and test data share similar joint probability distribution. TAL aims to build models that can perform tasks of target domain by learning knowledge from a semantic related but distribution different source domain. It is an energetic research filed of increasing influence and importance, which is presenting a blowout publication trend. This paper surveys the advances of TAL methodologies in the past decade, and the technical challenges and essential problems of TAL have been observed and discussed with deep insights and new perspectives. Broader solutions of transfer adaptation learning being created by researchers are identified, i.e., instance re-weighting adaptation, feature adaptation, classifier adaptation, deep network adaptation and adversarial adaptation, which are beyond the early semi-supervised and unsupervised split. The survey helps researchers rapidly but comprehensively understand and identify the research foundation, research status, theoretical limitations, future challenges and under-studied issues (universality, interpretability, and credibility) to be broken in the field toward universal representation and safe applications in open-world scenarios.

研究动机与目标

• 将 Transfer Adaptation Learning (TAL) 定义为迁移学习与领域自适应的统一视角。
• 明确主要的 TAL 方法学及其如何应对跨域的分布偏移。
• 讨论通用表征和在开放世界应用中的理论考量、挑战与未来方向。

提出的方法

• 提出一个包括实例重加权、特征自适应、分类器自适应、深度网络自适应和对抗自适应的 TAL 分类体系。
• 讨论弱监督学习视角，以及 SSL、AL、ZSL、OSR 与 TAL 的关系。
• 在每个 TAL 类别中，提出如分布差异最小化和领域对齐等核心方法机制。

实验结果

研究问题

• RQ1 哪些是解决跨域知识转移的核心 TAL 范式？
• RQ2 不同 TAL 方法如何缓解源域与目标域之间的分布不匹配？
• RQ3 通用 TAL 表现与开放世界可靠性面临的挑战与未来方向？

主要发现

• TAL 框架旨在同时最小化源风险、域差异和联合误差的组合，以界定目标风险。
• 识别出五个主要的 TAL 类别：实例重加权、特征自适应、分类器自适应、深度网络自适应和对抗自适应。
• 深度学习已成为 TAL 的主导，微调与对抗策略被视为核心技术。
• 将开集和部分域设定作为对传统闭集 DA 假设的扩展进行讨论。
• 本综述强调普适性、可解释性和可信度作为通用 TAL 表征中尚未充分研究的重要问题。