[Paper Review] A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets

Patryk Chrabąszcz, Ilya Loshchilov+1 more|arXiv (Cornell University)|Jul 27, 2017

Advanced Neural Network ApplicationsComputer Science9 references371 citations

TL;DR

The paper introduces downsampled ImageNet variants (ImageNet16x16/32x32/64x64) to enable faster experiments while preserving class count, shows similar hyperparameter regions across sizes, and demonstrates strong performance with Wide ResNets.

ABSTRACT

The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture search, and hyperparameter tuning) on the original dataset might be prohibitive, we propose to consider a downsampled version of ImageNet. In contrast to the CIFAR datasets and earlier downsampled versions of ImageNet, our proposed ImageNet32$ imes$32 (and its variants ImageNet64$ imes$64 and ImageNet16$ imes$16) contains exactly the same number of classes and images as ImageNet, with the only difference that the images are downsampled to 32$ imes$32 pixels per image (64$ imes$64 and 16$ imes$16 pixels for the variants, respectively). Experiments on these downsampled variants are dramatically faster than on the original ImageNet and the characteristics of the downsampled datasets with respect to optimal hyperparameters appear to remain similar. The proposed datasets and scripts to reproduce our results are available at http://image-net.org/download-images and https://github.com/PatrykChrabaszcz/Imagenet32_Scripts

Motivation & Objective

• Provide cheaper, scalable benchmarks by downsampling ImageNet while preserving the original number of classes and images.
• Evaluate whether downsampling preserves key learning dynamics and hyperparameter sensitivity.
• Assess how network width and learning rate interact with downsampled resolutions to guide cheap experimentation.

Proposed method

• Create ImageNet32x32, ImageNet64x64, and ImageNet16x16 by downsampling original ImageNet images while keeping class labels and image counts.
• Train Wide Residual Networks (WRN-N-k) with standard CIFAR-like configurations adapted for the downsampled images.
• Compare six downsampling methods (bicubic, bilinear, box, hamming, lanczos, nearest) and identify nearest neighbor as inferior.
• Use data augmentation (horizontal flips, random shifts) and standard SGD with momentum and scheduled learning rate drops.
• Evaluate performance across multiple network widths and downsampled resolutions to assess transferability to larger models.

Experimental results

Research questions

• RQ1 Does downsampling ImageNet to 32x32 (and variants 16x16/64x64) preserve the relative performance of different architectures and hyperparameters?
• RQ2 How does network width (k) interact with downsampled resolution to affect accuracy and training time?
• RQ3 Are results on downsampled ImageNet predictive of results on full ImageNet, enabling cheap architecture/hyperparameter search?
• RQ4 Which downsampling technique most effectively preserves information for classification on downsampled ImageNet?

Key findings

• Downsampling techniques yield similar results except nearest neighbor, which performs worse in all experiments.
• Wide ResNets achieve strong performance on ImageNet32x32, close to AlexNet’s original ImageNet results despite far fewer pixels per image.
• Increasing network width improves performance across all downsampling sizes; larger k yields better results.
• The optimal learning rate region is similar across ImageNet16x16, ImageNet32x32, and ImageNet64x64, and across different widths.
• The performance vs. training time tradeoff suggests using a combination of downsampling and network size for optimal anytime performance.
• Findings transfer plausibly to more expensive setups, enabling cheaper proxies for architecture/hyperparameter search.