[论文解读] Learning to Transfer Examples for Partial Domain Adaptation

Zhangjie Cao, Kaichao You+3 more|arXiv (Cornell University)|Mar 28, 2019

Domain Adaptation and Few-Shot Learning参考文献 43被引用 36

一句话总结

ETN 引入了可迁移性权重机制和一个辅助判别模块，以降低对离群源类的权重并通过一个可学习的端到端部分域自适应框架促进迁移。它在若干 PDA 基准上达到最先进结果。

ABSTRACT

Domain adaptation is critical for learning in new and unseen environments. With domain adversarial training, deep networks can learn disentangled and transferable features that effectively diminish the dataset shift between the source and target domains for knowledge transfer. In the era of Big Data, the ready availability of large-scale labeled datasets has stimulated wide interest in partial domain adaptation (PDA), which transfers a recognizer from a labeled large domain to an unlabeled small domain. It extends standard domain adaptation to the scenario where target labels are only a subset of source labels. Under the condition that target labels are unknown, the key challenge of PDA is how to transfer relevant examples in the shared classes to promote positive transfer, and ignore irrelevant ones in the specific classes to mitigate negative transfer. In this work, we propose a unified approach to PDA, Example Transfer Network (ETN), which jointly learns domain-invariant representations across the source and target domains, and a progressive weighting scheme that quantifies the transferability of source examples while controlling their importance to the learning task in the target domain. A thorough evaluation on several benchmark datasets shows that our approach achieves state-of-the-art results for partial domain adaptation tasks.

研究动机与目标

• Motivate partial domain adaptation (PDA) where source label space subsumes target label space and target labels are unknown during training.
• Propose ETN to jointly learn domain-invariant features and a progressive per-example transferability weight.
• Reduce negative transfer by down-weighting irrelevant source examples while enhancing positive transfer from shared classes.
• Leverage an auxiliary discriminative pathway to better quantify example transferability and improve PDA performance.

提出的方法

• Introduce a transferability weight w(xs) for each source example that scales its contribution to the source classifier and the domain discriminator.
• Use an auxiliary domain discriminator ˜Gd fed by an auxiliary label predictor ˜Gy with leaky-softmax to quantify transferability and distinguish shared vs outlier classes.
• Compute w(xs) = 1 - ˜Gd(Gf(xs)) with weights normalized within each mini-batch.
• Incorporate entropy minimization on target predictions to exploit unlabeled target data.
• Formulate a minimax objective that jointly optimizes feature extractor, source classifier, and domain discriminators, including auxiliary components.
• Train auxiliary components ˜Gy and ˜Gd to incorporate discriminative and domain information for better weighting.

实验结果

研究问题

• RQ1 How to effectively transfer knowledge from a large source label space to a smaller target label space in PDA?
• RQ2 Can a learned per-example transferability weight reduce negative transfer from outlier source classes while preserving positive transfer from shared classes?
• RQ3 Does incorporating discriminative information into auxiliary discriminators improve transferability quantification and PDA performance?
• RQ4 What is the impact of entropy minimization on leveraging unlabeled target data in PDA?

主要发现

• ETN achieves state-of-the-art results on Office-31, Office-Home, ImageNet-Caltech, and Caltech-256 PDA benchmarks.
• On Office-Home with ResNet-50, ETN averages 70.45% accuracy, outperforming prior methods like SAN (65.30) and PADA (62.06).
• On Office-31 and ImageNet-Caltech with ResNet-50, ETN attains averages of 79.08% and 79.08% respectively (per Table 2 and 3 context).
• ETN with ablations shows the full model outperforms variants lacking the classifier or auxiliary components, confirming the benefit of the transferability weighting and auxiliary discriminators.
• ETN consistently improves across multiple PDA task combinations, indicating robust transferability quantification and negative transfer mitigation.