[Paper Review] CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features

Sangdoo Yun, Dongyoon Han+4 more|arXiv (Cornell University)|May 13, 2019

Advanced Neural Network Applications51 references613 citations

TL;DR

CutMix replaces a patch of one training image with a patch from another image and mixes the labels proportionally to area, improving classification and localization with minimal overhead.

ABSTRACT

Regional dropout strategies have been proposed to enhance the performance of convolutional neural network classifiers. They have proved to be effective for guiding the model to attend on less discriminative parts of objects (e.g. leg as opposed to head of a person), thereby letting the network generalize better and have better object localization capabilities. On the other hand, current methods for regional dropout remove informative pixels on training images by overlaying a patch of either black pixels or random noise. Such removal is not desirable because it leads to information loss and inefficiency during training. We therefore propose the CutMix augmentation strategy: patches are cut and pasted among training images where the ground truth labels are also mixed proportionally to the area of the patches. By making efficient use of training pixels and retaining the regularization effect of regional dropout, CutMix consistently outperforms the state-of-the-art augmentation strategies on CIFAR and ImageNet classification tasks, as well as on the ImageNet weakly-supervised localization task. Moreover, unlike previous augmentation methods, our CutMix-trained ImageNet classifier, when used as a pretrained model, results in consistent performance gains in Pascal detection and MS-COCO image captioning benchmarks. We also show that CutMix improves the model robustness against input corruptions and its out-of-distribution detection performances. Source code and pretrained models are available at https://github.com/clovaai/CutMix-PyTorch .

Motivation & Objective

• Motivate improving generalization and localization in CNNs via regional dropout-based regularization.
• Develop a data augmentation that preserves informative pixels while enabling learning from partial object views.
• Demonstrate CutMix’s effectiveness across image classification, weakly supervised localization, and transfer learning tasks.
• Show robustness and uncertainty benefits from CutMix compared to other augmentations.

Proposed method

• Generate a new sample by combining two training images using a binary mask and mixed labels.
• Sample the mixing ratio lambda from a Beta(alpha, alpha) distribution (alpha=1 in experiments).
• Crop a region from one image and paste it into another image within a rectangular bounding box, with area proportional to 1 - lambda.
• Train with the CutMixed image and the mixed label using the original loss function.
• Optionally apply CutMix at the input image level or at higher feature levels (ablation studies).
• Keep training cost minimal with no extra computational overhead beyond standard augmentation.

Experimental results

Research questions

• RQ1 Does CutMix improve classification accuracy on large-scale datasets like ImageNet compared to Mixup and Cutout?
• RQ2 Does CutMix enhance weakly supervised localization by encouraging attention to broader object regions?
• RQ3 Do CutMix-pretrained models transfer better to downstream tasks such as object detection and image captioning?
• RQ4 Does CutMix improve robustness and calibration/uncertainty handling under adversarial or OOD conditions?

Key findings

• ImageNet: CutMix improves top-1 accuracy by +2.28% (ResNet-50) and +1.70% (ResNet-101) over baselines.
• CIFAR-100: CutMix achieves a state-of-the-art top-1 error of 14.47% (baseline 16.45% with PyramidNet-200), and yields notable gains versus Mixup and Cutout.
• Weakly supervised localization: CutMix improves WSOL accuracy by +5.4 percentage points on ImageNet and +0.9 on ImageNet localization, with substantial gains on CUB200-2011 as well.
• Transfer learning: Pretraining with CutMix improves downstream tasks; in Pascal VOC object detection (SSD/Faster R-CNN) and MS-COCO image captioning, CutMix-pretrained backbones provide measurable gains.
• Robustness/uncertainty: CutMix significantly enhances robustness to adversarial attacks (higher post-attack accuracy) and improves out-of-distribution detection metrics compared to Mixup and Cutout; it also reduces over-confidence tendencies.