[Paper Review] SNIPER: Efficient Multi-Scale Training

Bharat Singh, Mahyar Najibi+1 more|arXiv (Cornell University)|May 23, 2018

Advanced Neural Network ApplicationsComputer Science39 references55 citations

TL;DR

SNIPER trains detectors on small, multi-scale chips extracted around ground-truth instances and selected negatives, achieving near full image pyramid performance with about 3x efficiency and large batch training on a single GPU.

ABSTRACT

We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal network trained with a short learning schedule. Hence, the number of chips generated per image during training adaptively changes based on the scene complexity. SNIPER only processes 30% more pixels compared to the commonly used single scale training at 800x1333 pixels on the COCO dataset. But, it also observes samples from extreme resolutions of the image pyramid, like 1400x2000 pixels. As SNIPER operates on resampled low resolution chips (512x512 pixels), it can have a batch size as large as 20 on a single GPU even with a ResNet-101 backbone. Therefore it can benefit from batch-normalization during training without the need for synchronizing batch-normalization statistics across GPUs. SNIPER brings training of instance level recognition tasks like object detection closer to the protocol for image classification and suggests that the commonly accepted guideline that it is important to train on high resolution images for instance level visual recognition tasks might not be correct. Our implementation based on Faster-RCNN with a ResNet-101 backbone obtains an mAP of 47.6% on the COCO dataset for bounding box detection and can process 5 images per second during inference with a single GPU. Code is available at https://github.com/MahyarNajibi/SNIPER/.

Motivation & Objective

• Motivate efficient multi-scale training for instance-level visual recognition tasks.
• Propose a chip-based sampling strategy to reduce pixel processing while retaining performance.
• Enable large-batch training with batch normalization on a single GPU.
• Investigate the impact of context and scale on detector performance.
• Show competitive results on COCO using Faster-RCNN with a ResNet-101 backbone.

Proposed method

• Generate multi-scale image pyramids and sample 512×512 chips (context regions) around ground-truth objects (positive chips).
• Use a short-schedule region proposal network (RPN) to select negative chips covering likely false positives.
• Assign labels and bbox targets for proposals inside each chip without filtering by scale ranges. 0Train end-to-end like Faster-RCNN with chip-based inputs.
• Greedily cover ground-truth boxes with chips at appropriate scales to ensure all instances are observed. 0Retain cropped ground-truth boxes inside chips for label assignment.
• Train with large mini-batches using resampled chips at low resolution to enable batch normalization on a single GPU.
• Evaluate using standard COCO metrics and compare against SNIP on three scales; analyze recall and the effect of negative mining.

Experimental results

Research questions

• RQ1 Can training with small, scale-adapted chips at multiple scales match full image-pyramid multi-scale training performance?
• RQ2 Does negative chip mining improve detector performance while reducing computational cost?
• RQ3 Is large-batch training with batch normalization feasible on a single GPU when using SNIPER-style chips?
• RQ4 How does reducing context by chip-based sampling affect recall and mAP across object sizes?

Key findings

• SNIPER achieves comparable performance to full image pyramid multi-scale training while processing roughly 30% more pixels than single-scale training and significantly less than full pyramids.
• Training with negative chip mining improves AP from 43.4 to 46.1 for SNIPER with ResNet-101 and end-to-end training.
• Recall for RPN remains stable with or without negative sampling, indicating recall alone is not degraded by negative chip mining.
• Three-scale training (512, 1.667×, 3×) yields better performance than two-scale configurations, showing the benefit of multi-scale sampling.
• On COCO, SNIPER with ResNet-101 achieves 46.1 AP (and 47.6% with OpenImages pretraining and segmentation head), and can process about 5 images per second on a single V100 GPU.
• SNIPER enables training with large batches and BN on a single GPU, challenging the notion that high-resolution images are mandatory for instance-level recognition.