[Paper Review] Butterfly: A Panacea for All Difficulties in Wildly Unsupervised Domain Adaptation.

Feng Liu, Jie Lü+4 more|arXiv (Cornell University)|May 19, 2019

Domain Adaptation and Few-Shot Learning21 citations

TL;DR

This paper proposes Butterfly, a novel framework for wildly unsupervised domain adaptation (WUDA), where source domain data are noisy and unlabeled target data are used for adaptation. By simultaneously maintaining four models—two for noise correction and domain alignment, and two for target classification—Butterfly effectively handles noisy labels, domain shift, and distributional discrepancies, achieving state-of-the-art performance in WUDA settings.

ABSTRACT

In unsupervised domain adaptation (UDA), classifiers for the target domain (TD) are trained with clean labeled data from the source domain (SD) and unlabeled data from TD. However, in the wild, it is hard to acquire a large amount of perfectly clean labeled data in SD given limited budget. Hence, we consider a new, more realistic and more challenging problem setting, where classifiers have to be trained with noisy labeled data from SD and unlabeled data from TD---we name it wildly UDA (WUDA). We show that WUDA provably ruins all UDA methods if taking no care of label noise in SD, and to this end, we propose a Butterfly framework, a panacea for all difficulties in WUDA. Butterfly maintains four models (e.g., deep networks) simultaneously, where two take care of all adaptations (i.e., noisy-to-clean, labeled-to-unlabeled, and SD-to-TD-distributional) and then the other two can focus on classification in TD. As a consequence, Butterfly possesses all the necessary components for all the challenges in WUDA. Experiments demonstrate that under WUDA, Butterfly significantly outperforms existing baseline methods.

Motivation & Objective

• Address the realistic yet challenging setting of wildly unsupervised domain adaptation (WUDA), where source domain data are noisy and target domain data are unlabeled.
• Identify that existing UDA methods fail catastrophically under WUDA due to unaddressed label noise in the source domain.
• Develop a unified framework that simultaneously tackles label noise, domain shift, and distributional discrepancy in WUDA.
• Provide a robust solution that maintains high performance even when source data are imperfectly labeled, reflecting real-world data collection constraints.

Proposed method

• Butterfly maintains four deep neural networks in parallel: two for adaptation (noise correction and domain alignment), and two for target classification.
• The adaptation models jointly learn to correct noisy labels in the source domain and align source and target domain distributions.
• The framework uses a dual-branch architecture to decouple noise correction from classification, enabling stable and accurate learning.
• It leverages consistency regularization and self-training principles to refine pseudo-labels on target data and improve robustness.
• The four-model setup enables end-to-end optimization that jointly minimizes noise-induced error and domain shift.
• The method is designed to be agnostic to specific backbone architectures, allowing flexibility in implementation.

Experimental results

Research questions

• RQ1 Can existing unsupervised domain adaptation methods generalize to settings with noisy source labels, or do they fail catastrophically?
• RQ2 What architectural and training components are necessary to simultaneously address label noise, domain shift, and distributional discrepancy in WUDA?
• RQ3 Can a unified framework with multiple specialized models outperform existing methods in the more realistic WUDA setting?
• RQ4 How does the proposed Butterfly framework maintain performance under increasing levels of label noise in the source domain?

Key findings

• Butterfly significantly outperforms existing UDA baselines under the WUDA setting, demonstrating superior robustness to label noise.
• The framework effectively mitigates performance degradation caused by noisy source labels, which breaks standard UDA methods.
• The four-model architecture enables stable training and consistent improvements across multiple benchmark datasets.
• Butterfly achieves state-of-the-art accuracy on standard WUDA benchmarks, validating its effectiveness in handling multiple challenges simultaneously.
• Ablation studies confirm that both the noise correction and domain alignment components are essential for optimal performance.
• The method generalizes well across different backbone networks and data distributions, indicating strong robustness and adaptability.