[Paper Review] VisDA: The Visual Domain Adaptation Challenge

Xingchao Peng, Ben Usman+4 more|arXiv (Cornell University)|Oct 18, 2017

Domain Adaptation and Few-Shot Learning49 references575 citations

TL;DR

Introduces the VisDA2017 large-scale synthetic-to-real unsupervised domain adaptation benchmark for image classification and semantic segmentation, with baseline results and challenge-winning approaches showing substantial gains from domain adaptation methods.

ABSTRACT

We present the 2017 Visual Domain Adaptation (VisDA) dataset and challenge, a large-scale testbed for unsupervised domain adaptation across visual domains. Unsupervised domain adaptation aims to solve the real-world problem of domain shift, where machine learning models trained on one domain must be transferred and adapted to a novel visual domain without additional supervision. The VisDA2017 challenge is focused on the simulation-to-reality shift and has two associated tasks: image classification and image segmentation. The goal in both tracks is to first train a model on simulated, synthetic data in the source domain and then adapt it to perform well on real image data in the unlabeled test domain. Our dataset is the largest one to date for cross-domain object classification, with over 280K images across 12 categories in the combined training, validation and testing domains. The image segmentation dataset is also large-scale with over 30K images across 18 categories in the three domains. We compare VisDA to existing cross-domain adaptation datasets and provide a baseline performance analysis using various domain adaptation models that are currently popular in the field.

Motivation & Objective

• Address the domain shift problem by evaluating unsupervised domain adaptation (UDA) from synthetic to real imagery.
• Provide a large-scale, multi-domain benchmark for both object classification and semantic segmentation.
• Discourage reliance on target-domain labels or supervised pre-training in developing robust UDA methods.
• Offer baselines and challenge results to drive progress in cross-domain visual recognition.

Proposed method

• Construct a large-scale VisDA-C classification dataset with 152,397 synthetic training images and real validation/testing images from COCO and YouTube-BB across 12 categories.
• Compare baseline CNNs (AlexNet, ResNet/ResNext variants) to UDA methods such as Deep Adaptation Network (DAN) and Deep CORAL.
• Implement unsupervised domain adaptation using MMD-based and second-order statistics alignment techniques.
• Provide two target domains (validation: MS COCO; test: YouTube Bounding Boxes) to prevent hyperparameter tuning on the test set.
• Extend the benchmark with VisDA-S semantic segmentation using GTA5 (synthetic) to CityScapes (real) with Nexar as test domain.
• Offer baseline and challenge results to illustrate gains from domain adaptation and to motivate more robust UDA methods.

Experimental results

Research questions

• RQ1 How well can models trained on synthetic data adapt to unlabeled real-target domains in image classification?
• RQ2 What gains do standard UDA methods (e.g., DAN, Deep CORAL) achieve on VisDA-C compared to source-only baselines?
• RQ3 How does unsupervised domain adaptation perform for semantic segmentation across synthetic-to-real shifts (GTA5 to CityScapes, Nexar test)?
• RQ4 What design choices (validation split, lack of target labels, pre-training dependence) influence the difficulty and outcomes of cross-domain adaptation?
• RQ5 What directions can increase task difficulty to further stress-test UDA methods?

Key findings

• VisDA-C comprises over 280K images across 12 categories, with synthetic training data and real validation/testing data.
• Baseline source-only AlexNet on synthetic-to-real drops to 28.12% mean accuracy, highlighting substantial domain shift.
• DAN improves validation accuracy to 51.62% and Deep CORAL to 45.53% over the source-only baseline on VisDA-C.
• Top challenge results show further large gains (e.g., GFColourLabUEA achieving up to 92.8% on test) using semi-supervised and teacher-student strategies.
• On VisDA-S segmentation, adaptation from GTA5 to CityScapes raised mean IoU from 21.6 (source) to 25.5 (adapted) on validation, with competitive results on the Nexar test domain.
• The paper emphasizes reducing reliance on supervised pre-training (e.g., ImageNet) to reflect realistic deployment scenarios without similar-domain pre-training.