[Paper Review] Fixing the train-test resolution discrepancy: FixEfficientNet

Hugo Touvron, Andrea Vedaldi+2 more|arXiv (Cornell University)|Mar 18, 2020

Image Processing Techniques and ApplicationsEngineering16 references67 citations

TL;DR

The paper applies FixRes to EfficientNet to align train and test resolutions, producing FixEfficientNet which achieves top-tier ImageNet accuracy with a favorable parameter footprint, including 88.5% top-1 with L2 and 98.7% top-5.

ABSTRACT

This paper provides an extensive analysis of the performance of the EfficientNet image classifiers with several recent training procedures, in particular one that corrects the discrepancy between train and test images. The resulting network, called FixEfficientNet, significantly outperforms the initial architecture with the same number of parameters. For instance, our FixEfficientNet-B0 trained without additional training data achieves 79.3% top-1 accuracy on ImageNet with 5.3M parameters. This is a +0.5% absolute improvement over the Noisy student EfficientNet-B0 trained with 300M unlabeled images. An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88.5% top-1 accuracy (top-5: 98.7%), which establishes the new state of the art for ImageNet with a single crop. These improvements are thoroughly evaluated with cleaner protocols than the one usually employed for Imagenet, and particular we show that our improvement remains in the experimental setting of ImageNet-v2, that is less prone to overfitting, and with ImageNet Real Labels. In both cases we also establish the new state of the art.

Motivation & Objective

• Motivate to reduce train-test distribution mismatch caused by differing RoCs (Regions of Classification) and crops.
• Evaluate how applying FixRes fine-tuning to EfficientNet affects accuracy across model scales.
• Establish performance on ImageNet with and without external data under cleaner evaluation protocols.
• Assess generalization by evaluating on ImageNet-V2 and ImageNet Real Labels to validate robustness.

Proposed method

• Apply FixRes fine-tuning to EfficientNet architectures by re-training the classifier (or top layers) at the target resolution.
• Use FixRes data augmentation and combine with label smoothing during fine-tuning.
• Evaluate on ImageNet-2012 with single-crop top-1/top-5 metrics, starting from EfficientNet pre-trained models.
• Compare against EfficientNet baselines including Noisy Student and AdvProp variants without and with external data.
• Validate improvements under cleaner evaluation setups including ImageNet-V2 Matched Frequency and ImageNet Real Labels.

Experimental results

Research questions

• RQ1 How does FixRes fine-tuning affect EfficientNet performance across different model sizes (B0–L2) on ImageNet?
• RQ2 Does the FixEfficientNet approach improve generalization as measured on ImageNet-V2 and ImageNet Real Labels, beyond standard ImageNet-val results?
• RQ3 What is the impact of removing external data versus using 300M unlabeled images in combination with FixRes for EfficientNet?
• RQ4 How does FixEfficientNet compare to prior state-of-the-art EfficientNet variants (with Adversarial training or Noisy Student) under clean evaluation protocols?

Key findings

• FixEfficientNet-B0 achieves 79.3% Top-1 and 94.6% Top-5 on ImageNet with no extra data (vs 77.6%/93.3% for EfficientNet-B0).
• FixEfficientNet-L2 reaches 88.5% Top-1 and 98.7% Top-5 on ImageNet with extra data (and 88.5%/98.7% in Table 1 final row).
• On ImageNet without external data, FixEfficientNet outperforms EfficientNet with AdvProp across B0–B7, e.g., B7: 85.3% Top-1 (vs 85.2% for EfficientNet-B7).
• FixEfficientNet-L2 achieves new state-of-the-art single-crop results on ImageNet (Top-1 88.5%, Top-5 98.7%).
• With clean labels, FixEfficientNet-L2 attains 90.9% Top-1 and 98.8% Top-5, surpassing prior work.
• Across ImageNet-V2 Matched Frequency, FixEfficientNet variants show strong generalization, with Push beyond EfficientNet baselines.