Skip to main content
Nubint
QUICK REVIEW

[论文解读] CyCADA: Cycle-Consistent Adversarial Domain Adaptation

Judy Hoffman, Eric Tzeng|arXiv (Cornell University)|Nov 8, 2017
Domain Adaptation and Few-Shot Learning参考文献 35被引用 630
一句话总结

CyCADA 将循环一致的图像翻译与对抗域自适应结合，在像素和特征层面，以及语义一致性，来对不监督地跨域适配模型，在数字识别和语义分割中达到最先进的结果。

ABSTRACT

Domain adaptation is critical for success in new, unseen environments. Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts. Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at mapping images between domains, even without the use of aligned image pairs. We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model. CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs. Our model can be applied in a variety of visual recognition and prediction settings. We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

研究动机与目标

  • 动机并解决深度模型中合成/真实或不同视觉域之间的域迁移问题。
  • 开发一个无监督适配框架，在域翻译过程中保留语义内容。
  • 将像素空间和特征空间自适应与循环一致性和语义损失统一起来。
  • 在数字分类和城市场景语义分割任务上展示最先进的性能。
  • 展示图像空间自适应在性能提升的同时带来的可解释性优势。

提出的方法

  • 引入 G_S→T 和 G_T→S 生成器及对抗判别器，在源域和目标域之间进行映射。
  • 施加循环一致性损失，确保映射可逆且保留内容。
  • 通过使用固定的源任务模型来执行语义一致性损失，以在翻译后保持不变的标签。
  • 可选地包含特征级GAN损失，以对齐中间表示。
  • 优化联合的 CyCADA 目标，结合任务损失、GAN 损失、循环损失和语义损失。
  • 实现为像素对像素卷积网络用于 G，以及用于 f 和 D 的 FCN/卷积网络。
Figure 1: We propose CyCADA, an adversarial unsupervised adaptation algorithm which uses cycle and semantic consistency to perform adaptation at multiple levels in a deep network. Our model provides significant performance improvements over source model baselines.
Figure 1: We propose CyCADA, an adversarial unsupervised adaptation algorithm which uses cycle and semantic consistency to perform adaptation at multiple levels in a deep network. Our model provides significant performance improvements over source model baselines.

实验结果

研究问题

  • RQ1循环一致的对抗映射在跨域翻译时能否保留语义内容？
  • RQ2多级自适应（像素级和特征级）是否比单级方法更能提升无监督域自适应？
  • RQ3在域移位下，循环一致性和语义一致性如何影响数字识别和语义分割的性能？
  • RQ4除了特征空间自适应外，图像空间（像素）自适应对可解释性和性能有何影响？
  • RQ5CyCADA 在合成到现实场景中弥合源训练与目标训练性能差距的程度如何？

主要发现

模型MNIST → USPSUSPS → MNISTSVHN → MNIST
仅源82.2 ± 0.869.6 ± 3.867.1 ± 0.6
DANN (Ganin et al., 2016)--73.6
DTN (Taigman et al., 2017a)--84.4
CoGAN (Liu & Tuzel, 2016a)91.289.1-
ADDA (Tzeng et al., 2017)89.4 ± 0.290.1 ± 0.876.0 ± 1.8
CyCADA pixel only95.6 ± 0.296.4 ± 0.170.3 ± 0.2
CyCADA pixel+feat95.6 ± 0.296.5 ± 0.190.4 ± 0.4
Target only96.3 ± 0.199.2 ± 0.199.2 ± 0.1
  • CyCADA 在 USPS、MNIST 与 SVHN 转变上的数字适应方面达到最先进的结果。
  • 单独的像素空间自适应在较小域移时取得强劲结果，添加特征空间自适应在较大移位时提供进一步提升。
  • 在语义分割中，CyCADA 在合成到真实任务和跨季节自适应中达到最先进的性能，在若干指标接近目标监督的 oracle 水平。
  • 像素空间和特征空间自适应的联合在数字分类任务中提供最佳整体性能。
  • 图像空间自适应提供可解释的可视化翻译（如 SYNTHIA 的秋季到冬季），与改进的分割性能相关。
Figure 2: Cycle-consistent adversarial adaptation of pixel-space inputs. By directly remapping source training data into the target domain, we remove the low-level differences between the domains, ensuring that our task model is well-conditioned on target data. We depict here the image-level GAN los
Figure 2: Cycle-consistent adversarial adaptation of pixel-space inputs. By directly remapping source training data into the target domain, we remove the low-level differences between the domains, ensuring that our task model is well-conditioned on target data. We depict here the image-level GAN los

更好的研究，从现在开始

从论文设计到论文写作，大幅缩短您的研究时间。

无需绑定信用卡

本解读由 AI 生成，并经人工编辑审核。