[论文解读] A Comprehensive Survey on Transfer Learning

Fuzhen Zhuang, Zhiyuan Qi+6 more|arXiv (Cornell University)|Nov 7, 2019

Domain Adaptation and Few-Shot Learning参考文献 208被引用 228

一句话总结

本综述系统性地评估了超过 forty 种迁移学习方法，聚焦于从数据和模型角度的同质迁移学习，并在 three datasets 上对 twenty 多种模型进行比较，以为实践提供指导。

ABSTRACT

Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prospects, transfer learning has become a popular and promising area in machine learning. Although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances in transfer learning. Due to the rapid expansion of the transfer learning area, it is both necessary and challenging to comprehensively review the relevant studies. This survey attempts to connect and systematize the existing transfer learning researches, as well as to summarize and interpret the mechanisms and the strategies of transfer learning in a comprehensive way, which may help readers have a better understanding of the current research status and ideas. Unlike previous surveys, this survey paper reviews more than forty representative transfer learning approaches, especially homogeneous transfer learning approaches, from the perspectives of data and model. The applications of transfer learning are also briefly introduced. In order to show the performance of different transfer learning models, over twenty representative transfer learning models are used for experiments. The models are performed on three different datasets, i.e., Amazon Reviews, Reuters-21578, and Office-31. And the experimental results demonstrate the importance of selecting appropriate transfer learning models for different applications in practice.

研究动机与目标

• 提供对迁移学习及其机制的统一、全面视角。
• 从数据和模型角度系统性地对现有迁移学习方法进行分类。
• 总结减少跨域分布差异并保持数据属性的策略。
• 通过在标准基准上比较多种迁移学习模型，提供实际指导。

提出的方法

• 评审超过 forty 种代表性迁移学习方法，重点关注同质迁移学习。
• 从数据层面和模型层面解释迁移学习。
• 讨论实例加权、分布度量（如 MMD）和特征变换技术。
• 描述特征增强方法（如 FAM）及其在异构任务中的局限性。
• 通过在 three datasets（Amazon Reviews、Reuters-21578、Office-31）上评估 twenty 多种模型，概括实验做法。

实验结果

研究问题

• RQ1 跨领域转移知识的主要数据导向和模型导向策略是什么？
• RQ2 同质和异质迁移情景有何区别，分别采用了哪些方法？
• RQ3 哪些迁移学习方法在标准基准和特定设置下表现良好？
• RQ4 如何衡量并缓解源域与目标域之间的分布差异？

主要发现

• 迁移学习方法大致可分为实例型、特征型、参数型和关系型等类别。
• 最大均值差异（MMD）及相关度量常用于量化分布差异并指导适应。
• 特征增强和映射技术实现跨域的共同潜在表示，存在适用于同质和异质情形的如 FAM 和 HFA 的改进。
• 实证结果强调在标准数据集上选择合适的迁移学习模型对具体应用的重要性。
• 综述指出当领域相关性较弱或不对齐时，可能发生负迁移。