[Paper Review] Do CIFAR-10 Classifiers Generalize to CIFAR-10?

Benjamin Recht, Rebecca Roelofs+2 more|arXiv (Cornell University)|Jun 1, 2018

Advanced Neural Network ApplicationsComputer Science2 references133 citations

TL;DR

The paper creates a new truly unseen CIFAR-10 test set and shows a large drop in accuracy (4–10%) across a broad range of models, while relative model rankings remain similar; it argues the drop reflects benign distribution shift rather than test-set overfitting.

ABSTRACT

Machine learning is currently dominated by largely experimental work focused on improvements in a few key tasks. However, the impressive accuracy numbers of the best performing models are questionable because the same test sets have been used to select these models for multiple years now. To understand the danger of overfitting, we measure the accuracy of CIFAR-10 classifiers by creating a new test set of truly unseen images. Although we ensure that the new test set is as close to the original data distribution as possible, we find a large drop in accuracy (4% to 10%) for a broad range of deep learning models. Yet more recent models with higher original accuracy show a smaller drop and better overall performance, indicating that this drop is likely not due to overfitting based on adaptivity. Instead, we view our results as evidence that current accuracy numbers are brittle and susceptible to even minute natural variations in the data distribution.

Motivation & Objective

• Assess whether CIFAR-10 classifiers overfit to the original test set through repeated evaluation.
• Create a new test set with a distribution closely matching CIFAR-10 to measure true generalization.
• Quantify the accuracy gap between original and new test sets across a wide range of models.
• Investigate potential causes of the gap and whether hyperparameter tuning or data leakage explains it.
• Discuss implications for evaluating progress in ML and generalization under distribution shift.

Proposed method

• Curate a new 2,000–4,000 image test set whose sub-class distribution matches CIFAR-10 by selecting images from the same Tiny Images keywords.
• Evaluate 30 image classifiers spanning years of CIFAR-10 research on both the original and the new test set.
• Analyze the accuracy gap and model ranking changes with linear fit and simple mixture-distribution reasoning.
• Test hypotheses including statistical error, near-duplicate removal effects, hyperparameter tuning, and hard-image inspection.
• Conduct cross-validation and light retraining experiments to probe distribution similarity between datasets.

Experimental results

Research questions

• RQ1 Does a newly collected, truly unseen CIFAR-10 test set reveal lower accuracy for CIFAR-10 classifiers compared to the original test set?
• RQ2 Are the relative rankings of models stable when evaluated on the new test set?
• RQ3 What mechanisms (e.g., distribution shift, hyperparameter tuning) explain the accuracy gap between the original and new test sets?
• RQ4 Does model retraining or cross-validation on the original data predict performance on the new test set?

Key findings

• A substantial absolute drop in accuracy is observed for all deep models when evaluated on the new test set (e.g., VGG/ResNet drop from ~93% to ~85%; Shake-Shake top model from 97% to ~93%).
• The relative ordering of models is largely preserved; a linear relationship approximately maps orig accuracy to new accuracy (acc_new ≈ 1.62·acc_orig − 65.51%).
• Some techniques (e.g., Cutout augmentation) yield larger gains on the new test set than on the original, indicating differential transfer effects.
• Hyperparameter tuning yields at most modest gains on the new test set (largest ~0.6–1% improvement in their experiments).
• The observed gap is not attributable to simple statistical error or extensive near-duplicate leakage; there is evidence supporting a benign distribution shift rather than test-set overfitting.
• Cross-validation on the original CIFAR-10 data produced stable predictions for the new test set, suggesting the shift is not due to dramatic distributional changes.