[Paper Review] Measuring Robustness to Natural Distribution Shifts in Image Classification

Rohan Taori, Achal Dave+4 more|arXiv (Cornell University)|Jul 1, 2020

Anomaly Detection Techniques and ApplicationsComputer Science99 references170 citations

TL;DR

This paper evaluates 204 ImageNet models across 213 natural distribution shifts to assess whether robustness learned on synthetic shifts transfers to real-world, natural shifts and finds that it largely does not; training on more diverse data helps modestly, but robustness to natural shifts remains an open problem.

ABSTRACT

We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets. Most research on robustness focuses on synthetic image perturbations (noise, simulated weather artifacts, adversarial examples, etc.), which leaves open how robustness on synthetic distribution shift relates to distribution shift arising in real data. Informed by an evaluation of 204 ImageNet models in 213 different test conditions, we find that there is often little to no transfer of robustness from current synthetic to natural distribution shift. Moreover, most current techniques provide no robustness to the natural distribution shifts in our testbed. The main exception is training on larger and more diverse datasets, which in multiple cases increases robustness, but is still far from closing the performance gaps. Our results indicate that distribution shifts arising in real data are currently an open research problem. We provide our testbed and data as a resource for future work at https://modestyachts.github.io/imagenet-testbed/ .

Motivation & Objective

• Assess how current ImageNet models handle natural distribution shifts arising from real-world variations.
• Distinguish robustness to natural shifts from robustness to synthetic perturbations.
• Quantify the relationship between standard accuracy and robustness under shift, controlling for accuracy.
• Identify robustness interventions that meaningfully improve performance on natural shifts.
• Provide a large open testbed and dataset to guide future robustness research.

Proposed method

• Evaluate 204 pre-trained ImageNet models under 213 test conditions covering natural and synthetic shifts.
• Define two-test-set framework with standard (S1) and shifted (S2) accuracies for each model.
• Introduce effective robustness: ρ(f) = acc2(f) − β(acc1(f)), where β is a baseline mapping from standard accuracy to shifted-accuracy.
• Define relative robustness τ(f′) = acc2(f′) − acc2(f) to measure intervention impact.
• Classify shifts into natural (consistency, dataset shifts, adversarially filtered) and synthetic (corruptions, style transfer, adversarial examples).
• Analyze correlations between synthetic robustness metrics and natural robustness; assess training data scale effects on robustness.

Experimental results

Research questions

• RQ1 Does robustness gained from synthetic distribution shifts transfer to robustness against natural distribution shifts?
• RQ2 Do robustness interventions that improve synthetic robustness also improve robustness on natural shifts?
• RQ3 How does training on larger or more diverse datasets affect robustness to natural shifts?
• RQ4 Can we define a metric (effective robustness) that separates standard accuracy from robustness gains?
• RQ5 Which natural distribution shifts are most predictive of improvements from robustness interventions?

Key findings

• Robustness interventions largely fail to improve performance on natural distribution shifts in the testbed.
• Training on substantially more diverse data yields small but detectable gains in robustness to some dataset shifts, though gains are data-size dependent and not universal.
• Model accuracy on the standard test set strongly predicts accuracy under natural distribution shifts (high r2 on several shifts, e.g., ImageNetV2, ObjectNet).
• Adversarially trained (Lp) models show notable effective robustness to consistency shifts (ImageNet-Vid-Robust and YTBB-Robust) but often have negative relative robustness, except for a few exceptions.
• ImageNet-A exhibits a knee: models with higher standard accuracy improve much more on ImageNet-A than lower-accuracy models, suggesting adversarial filtering creates qualitatively different shifts.
• Aggregated synthetic robustness measures (image corruptions, PGD attacks) are only weakly predictive of effective robustness on natural shifts (correlations near 0–0.24).
• Some models trained on massive data (e.g., EfficientNet-L2 NoisyStudent, ResNet152 on 11k+ classes) show positive effective robustness, but this is not universal across all high-data models.