[Paper Review] Towards Corruption-Agnostic Robust Domain Adaptation

Yifan Xu, Kekai Sheng+4 more|arXiv (Cornell University)|Apr 21, 2021

Domain Adaptation and Few-Shot Learning44 references4 citations

TL;DR

This paper proposes Corruption-Agnostic Robust Domain Adaptation (CRDA), a novel framework that enhances domain adaptation models to be robust against unseen corruptions while maintaining high accuracy on clean data. It introduces the Domain Discrepancy Generator (DDG) to synthesize samples maximizing domain shift, mimicking unpredictable corruptions, and a teacher-student contrastive learning scheme to stabilize training and improve robustness, outperforming baselines in both original accuracy and corruption robustness.

ABSTRACT

Big progress has been achieved in domain adaptation in decades. Existing works are always based on an ideal assumption that testing target domain are i.i.d. with training target domains. However, due to unpredictable corruptions (e.g., noise and blur) in real data like web images, domain adaptation methods are increasingly required to be corruption robust on target domains. In this paper, we investigate a new task, Corruption-agnostic Robust Domain Adaptation (CRDA): to be accurate on original data and robust against unavailable-for-training corruptions on target domains. This task is non-trivial due to large domain discrepancy and unsupervised target domains. We observe that simple combinations of popular methods of domain adaptation and corruption robustness have sub-optimal CRDA results. We propose a new approach based on two technical insights into CRDA: 1) an easy-to-plug module called Domain Discrepancy Generator (DDG) that generates samples that enlarge domain discrepancy to mimic unpredictable corruptions; 2) a simple but effective teacher-student scheme with contrastive loss to enhance the constraints on target domains. Experiments verify that DDG keeps or even improves performance on original data and achieves better corruption robustness that baselines.

Motivation & Objective

• To address the gap in domain adaptation models that fail under real-world data corruptions despite strong performance on clean data.
• To develop a method that maintains high accuracy on original target domains while achieving robustness against unseen, unpredictable corruptions.
• To unify domain shift adaptation and corruption robustness in an unsupervised setting without requiring corruption-specific training data.
• To overcome the limitations of combining existing domain adaptation and corruption robustness methods, which yield sub-optimal results due to domain discrepancy and weak supervision on target domains.

Proposed method

• Proposes the Domain Discrepancy Generator (DDG), a plug-in module that generates synthetic data samples by maximizing domain discrepancy between source and target domains, simulating unpredictable real-world corruptions.
• Uses a gradient-based optimization strategy to identify and generate samples that induce the largest domain shift, grounded in theoretical analysis of the δ-cycle boundary in feature space.
• Introduces a teacher-student warm-up scheme with contrastive loss to stabilize early training and improve feature generalization on unlabeled target domains.
• Employs contrastive learning to align features of clean and corrupted versions of images, enabling order-invariant representations across corruption severity levels.
• Trains a student model to distill robust features from a pre-trained teacher model, using DDG-generated samples as positive pairs in contrastive loss.
• Applies the method end-to-end in an unsupervised domain adaptation setting, with no need for labeled corrupted data during training.

Figure 1. Instances of corruption-agnostic robust domain adaptation (CRDA). To our surprise, existing domain adaptation method (CDAN+TN (Wang et al . , 2019 ) ) and corruption robust method (AugMix (Hendrycks et al . , 2020b ) ), even their combination, suffer from unseen corruptions on target domai

Experimental results

Research questions

• RQ1 Can domain adaptation models be made robust to unseen corruptions without access to corruption-specific training data?
• RQ2 How can domain discrepancy be leveraged to generate realistic, corruption-like perturbations that improve robustness?
• RQ3 Does a teacher-student contrastive learning scheme enhance robustness and generalization on unlabeled target domains in domain adaptation?
• RQ4 Can a unified framework simultaneously improve performance on clean data and robustness against diverse corruptions?
• RQ5 Is there a trade-off between accuracy on clean data and robustness to corruptions in domain adaptation?

Key findings

• The proposed CRDA method achieves state-of-the-art performance on both clean data and corrupted data, with a 23.0% error rate on the most severe contrast corruption in the Office-Home dataset under the Ar→Rw setting.
• The DDG module improves corruption robustness significantly compared to baselines, reducing error rates by up to 20% on severe corruptions like glass blur and JPEG compression.
• The teacher-student warm-up scheme improves both original accuracy and robustness, achieving 21.2% error on clean data and 23.0% on contrast corruption, outperforming data augmentation baselines.
• Abllation studies show that setting the update stride η > 2δ in DDG, which focuses on edge points of the δ-cycle, yields the best performance, confirming theoretical insights.
• The model learns order-invariant representations: even when trained only on a single corruption severity, it generalizes well across all severity levels, as shown by consistent feature distance trends.
• DDG maintains or improves original domain adaptation accuracy while adding robustness, avoiding the typical accuracy-robustness trade-off.

Figure 2. Examples of $15$ kinds of corruptions.