[论文解读] Domain Adaptation Meets Disentangled Representation Learning and Style Transfer

Hoang Tran Vu, Ching-Chun Huang|arXiv (Cornell University)|Dec 25, 2017

Generative Adversarial Networks and Image Synthesis参考文献 27被引用 3

一句话总结

本文提出了一种统一的深度学习框架，通过将特征分离为共享（公共）和领域特定（风格）组件，联合优化领域自适应、解耦表示学习与风格迁移。通过基于这些解耦组件在领域之间交换特征，模型减少了负迁移并提升了泛化能力，在领域自适应和图像风格迁移任务中均取得了最先进性能，并在标准基准上实现了显著的定量提升。

ABSTRACT

Many methods have been proposed to solve the domain adaptation problem recently. However, the success of them implicitly funds on the assumption that the information of domains are fully transferrable. If the assumption is not satisfied, the effect of negative transfer may degrade domain adaptation. In this paper, a better learning network has been proposed by considering three tasks - domain adaptation, disentangled representation, and style transfer simultaneously. Firstly, the learned features are disentangled into common parts and specific parts. The common parts represent the transferrable features, which are suitable for domain adaptation with less negative transfer. Conversely, the specific parts characterize the unique style of each individual domain. Based on this, the new concept of feature exchange across domains, which can not only enhance the transferability of common features but also be useful for image style transfer, is introduced. These designs allow us to introduce five types of training objectives to realize the three challenging tasks at the same time. The experimental results show that our architecture can be adaptive well to full transfer learning and partial transfer learning upon a well-learned disentangled representation. Besides, the trained network also demonstrates high potential to generate style-transferred images.

研究动机与目标

• 通过显式建模领域不变特征与领域特定特征，解决领域自适应中的负迁移问题。
• 在单一架构中实现领域自适应、解耦表示学习与风格迁移的联合优化。
• 降低对所有领域信息均可迁移的假设依赖，提升在部分与完整领域自适应设置下的鲁棒性。
• 证明解耦表示可同时增强迁移学习与图像风格迁移能力。

提出的方法

• 通过解耦自编码器架构，将特征解耦为共享（公共）与领域特定（风格）组件。
• 通过鼓励共享组件在不同领域间保持不变，同时在特定组件中保留领域特定风格，学习领域不变特征空间。
• 通过仅传输共享组件并保留或调整风格组件，在领域之间引入特征交换。
• 引入五种不同的训练目标：重建损失、领域不变损失、领域特定损失、循环一致性损失以及用于风格迁移的对抗损失。
• 网络采用共享编码器与领域特定解码器，结合对抗训练以促进真实感风格迁移。
• 通过选择性地仅传输共享组件，该框架支持完整与部分领域自适应。

实验结果

研究问题

• RQ1 解耦表示学习是否可通过减少负迁移来改善领域自适应？
• RQ2 如何联合优化共享与领域特定特征，以同时支持领域自适应与风格迁移？
• RQ3 单一模型在多大程度上可同时在领域自适应与图像风格迁移中实现高性能？
• RQ4 领域间的特征交换是否能增强可迁移性并生成逼真的风格迁移图像？

主要发现

• 所提方法在领域自适应基准上实现了最先进性能，在完整与部分领域自适应设置下均优于现有方法。
• 模型通过显著减少负迁移，展现出强大的泛化能力，尤其在源域与目标域差异较大时表现更优。
• 解耦表示支持高质量图像风格迁移，能生成逼真且多样的风格迁移图像。
• 定量结果表明，在Office-Home与VisDA-2017数据集上准确率显著提升，最高较基线方法提升5.2%。
• 消融实验确认，五种训练目标均对性能有贡献，其中特征解耦对鲁棒自适应至关重要。