[Paper Review] Scalable, High-Quality Object Detection

Christian Szegedy, Scott Reed+3 more|arXiv (Cornell University)|Dec 3, 2014

Advanced Neural Network Applications29 references337 citations

TL;DR

This paper proposes Multi-scale Convolutional MultiBox (MSC-MultiBox), a learnable, scalable object proposal method that uses deep convolutional networks with multi-scale predictors to generate high-quality, ranked region proposals. It achieves state-of-the-art mAP of 0.50 with a single model and 0.52 with an ensemble on ILSVRC 2014, outperforming hand-engineered methods like Selective Search and MCG while enabling efficient runtime-quality trade-offs through learnable proposal ranking.

ABSTRACT

Current high-quality object detection approaches use the scheme of salience-based object proposal methods followed by post-classification using deep convolutional features. This spurred recent research in improving object proposal methods. However, domain agnostic proposal generation has the principal drawback that the proposals come unranked or with very weak ranking, making it hard to trade-off quality for running time. This raises the more fundamental question of whether high-quality proposal generation requires careful engineering or can be derived just from data alone. We demonstrate that learning-based proposal methods can effectively match the performance of hand-engineered methods while allowing for very efficient runtime-quality trade-offs. Using the multi-scale convolutional MultiBox (MSC-MultiBox) approach, we substantially advance the state-of-the-art on the ILSVRC 2014 detection challenge data set, with $0.5$ mAP for a single model and $0.52$ mAP for an ensemble of two models. MSC-Multibox significantly improves the proposal quality over its predecessor MultiBox~method: AP increases from $0.42$ to $0.53$ for the ILSVRC detection challenge. Finally, we demonstrate improved bounding-box recall compared to Multiscale Combinatorial Grouping with less proposals on the Microsoft-COCO data set.

Motivation & Objective

• To close the performance gap between learned and hand-engineered object proposal methods by leveraging deep learning for end-to-end proposal generation.
• To enable efficient runtime-quality trade-offs in object detection by learning a ranked proposal system that allows threshold-based filtering for speed or accuracy.
• To improve detection performance by integrating contextual modeling and multi-scale convolutional predictors into a scalable, class-agnostic proposal framework.
• To demonstrate that data-driven proposal generation can surpass traditional salience-based methods like MCG and Selective Search in both quality and efficiency.

Proposed method

• The method employs a multi-scale convolutional architecture based on Inception-v3 to predict bounding box coordinates and confidence scores across multiple feature map scales.
• It uses a class-agnostic proposal generator that produces region proposals without requiring class-specific tuning, improving scalability and generalization.
• Proposals are generated via multi-crop inference, where the network processes the full image and multiple sliding crops at different scales, followed by non-maximum suppression with a 0.85 IoU threshold.
• A contextual modeling component is integrated during post-classification to improve detection accuracy by leveraging spatial context around proposals.
• The system uses asynchronous stochastic gradient descent with 30 model replicas to train the proposal network on 2 million batches of size 32.
• Ensembling of two MultiBox generators and three post-classifiers is applied to further boost performance, especially at high-quality detection settings.

Experimental results

Research questions

• RQ1 Can a purely learned, end-to-end deep learning approach match or surpass the performance of hand-engineered salience-based proposal methods like Selective Search and MCG?
• RQ2 Can multi-scale convolutional predictors significantly improve proposal quality and detection mAP compared to single-scale or non-learned methods?
• RQ3 Does integrating contextual modeling during post-classification lead to measurable improvements in detection accuracy?
• RQ4 Can a learnable proposal system enable flexible runtime-quality trade-offs by adjusting proposal confidence thresholds?
• RQ5 How does the proposed MSC-MultiBox method compare in recall and mAP to state-of-the-art methods like MCG on standard benchmarks such as COCO and ImageNet?

Key findings

• MSC-MultiBox achieves a mean average precision (mAP) of 0.50 on the ILSVRC 2014 detection challenge with a single model, setting a new state of the art.
• With an ensemble of two MultiBox generators and three post-classifiers, the method reaches 0.52 mAP, significantly outperforming prior state-of-the-art methods.
• On the Microsoft COCO validation set, MSC-MultiBox achieves higher per-class average recall than MCG at overlap thresholds up to 0.75, especially with fewer than 200 proposals per image.
• The proposal quality improves from AP 0.42 in the original MultiBox to 0.53 in MSC-MultiBox on the ILSVRC 2014 dataset, demonstrating a substantial gain in proposal quality.
• The method reduces computational cost while maintaining or improving detection performance, showing that learned proposal generation can replace expensive hand-engineered alternatives.
• Even with only 15 proposals per image, the method exceeds the previous SOTA performance on ILSVRC 2014, demonstrating high efficiency and effectiveness.