[Paper Review] Rethinking Distributional Matching Based Domain Adaptation

Bo Li, Yezhen Wang+7 more|arXiv (Cornell University)|Jun 23, 2020

Domain Adaptation and Few-Shot LearningComputer Science54 references35 citations

TL;DR

The paper analyzes limitations of distributional matching (DM) in domain adaptation under realistic domain shifts (RDS) and proposes InstaPBM, an instance-based predictive behavior matching method, to achieve robust unsupervised domain adaptation across both conventional and RDS benchmarks.

ABSTRACT

Domain adaptation (DA) is a technique that transfers predictive models trained on a labeled source domain to an unlabeled target domain, with the core difficulty of resolving distributional shift between domains. Currently, most popular DA algorithms are based on distributional matching (DM). However in practice, realistic domain shifts (RDS) may violate their basic assumptions and as a result these methods will fail. In this paper, in order to devise robust DA algorithms, we first systematically analyze the limitations of DM based methods, and then build new benchmarks with more realistic domain shifts to evaluate the well-accepted DM methods. We further propose InstaPBM, a novel Instance-based Predictive Behavior Matching method for robust DA. Extensive experiments on both conventional and RDS benchmarks demonstrate both the limitations of DM methods and the efficacy of InstaPBM: Compared with the best baselines, InstaPBM improves the classification accuracy respectively by $4.5\%$, $3.9\%$ on Digits5, VisDA2017, and $2.2\%$, $2.9\%$, $3.6\%$ on DomainNet-LDS, DomainNet-ILDS, ID-TwO. We hope our intuitive yet effective method will serve as a useful new direction and increase the robustness of DA in real scenarios. Code will be available at anonymous link: https://github.com/pikachusocute/InstaPBM-RobustDA.

Motivation & Objective

• Examine limitations of distributional matching (DM) methods under realistic domain shifts (RDS).
• Create benchmarks that reflect real-world domain shifts (LDS, ILDS, TwO).
• Propose a robust domain adaptation method that does not rely on DM assumptions.
• Introduce InstaPBM to align predictive behaviors across source and target domains.

Proposed method

• Critically analyze DM-based UDA and identify failure modes under LDS, ILDS, and TwO.
• Introduce RDS benchmarks: LDS, ILDS, TwO for rigorous evaluation.
• Propose InstaPBM consisting of three components: Mutual Information Predictive Behavior Matching (MIM), Contrastive Predictive Behavior Matching, and Mix-up Predictive Behavior Matching, plus self-supervised and mix-up tasks.
• Define predictive behaviors as properties of pθ(y|x) that should hold across domains and do not rely on distributional alignment.
• Develop training objectives: L_M for mutual information maximization, L_C for contrastive PB matching, L_U for mix-up PBM, and L_S for self-supervised task-oriented PBM.
• Provide an integrated optimization (Eq. 7) combining components with corresponding weights.

Experimental results

Research questions

• RQ1 Can DM-based domain adaptation methods guarantee target-label correctness under realistic domain shifts?
• RQ2 Do new RDS benchmarks reveal systematic limitations of DM methods?
• RQ3 Can matching predictive behaviors of the classifier across domains yield robust unsupervised domain adaptation?
• RQ4 How effective is InstaPBM, and which components contribute most to performance and robustness?

Key findings

• InstaPBM consistently outperforms state-of-the-art DM baselines on conventional benchmarks (Digits5, VisDA2017) and RDS benchmarks.
• On Digits5, InstaPBM improves accuracy by 4.5 percentage points over the best baseline; on VisDA2017, by 3.9 points.
• On DomainNet-LDS, DomainNet-ILDS, and ID-TwO, InstaPBM yields improvements of 2.2–3.6 points over baselines.
• InstaPBM shows stronger robustness to realistic domain shifts than DM methods across LDS, ILDS, and TwO benchmarks.
• Ablation shows Mutual Information PB Matching (MIM) provides the largest single-component gain, followed by CPBM_ALL.