[Paper Review] DetNAS: Backbone Search for Object Detection

Yukang Chen, Tong Yang+4 more|arXiv (Cornell University)|Mar 26, 2019

Advanced Neural Network ApplicationsComputer Science44 references174 citations

TL;DR

DetNAS introduces a three-step backbone search framework using a one-shot supernet and evolutionary search to tailor backbones for object detectors, achieving higher COCO mmAP with fewer FLOPs than hand-crafted networks.

ABSTRACT

Object detectors are usually equipped with backbone networks designed for image classification. It might be sub-optimal because of the gap between the tasks of image classification and object detection. In this work, we present DetNAS to use Neural Architecture Search (NAS) for the design of better backbones for object detection. It is non-trivial because detection training typically needs ImageNet pre-training while NAS systems require accuracies on the target detection task as supervisory signals. Based on the technique of one-shot supernet, which contains all possible networks in the search space, we propose a framework for backbone search on object detection. We train the supernet under the typical detector training schedule: ImageNet pre-training and detection fine-tuning. Then, the architecture search is performed on the trained supernet, using the detection task as the guidance. This framework makes NAS on backbones very efficient. In experiments, we show the effectiveness of DetNAS on various detectors, for instance, one-stage RetinaNet and the two-stage FPN. We empirically find that networks searched on object detection shows consistent superiority compared to those searched on ImageNet classification. The resulting architecture achieves superior performance than hand-crafted networks on COCO with much less FLOPs complexity.

Motivation & Objective

• Motivate the need for backbones specifically designed for object detection rather than image classification backbones.
• Propose a practical NAS framework that decouples weight training from architecture search via a one-shot supernet.
• Show that backbones searched on object detection outperform those searched on ImageNet classification across detectors and datasets.
• Demonstrate that DetNASNet and DetNASNet (3.8) achieve superior accuracy with lower computational cost on COCO and VOC.

Proposed method

• Construct a one-shot supernet that encompasses all candidate backbones in the search space.
• Pre-train the supernet on ImageNet with a path-wise sampling strategy to reflect relative architecture performance.
• Fine-tune the supernet on detection datasets (COCO/VOC) with SyncBN to handle small batch statistics during fine-tuning.
• Search architectures on the trained supernet using an evolutionary algorithm under FLOPs/inference constraints.
• Recompute batch statistics for each evaluated path to ensure valid BN statistics for BN layers during evaluation.

Experimental results

Research questions

• RQ1 Can a backbone searched directly on object detection outperform backbones searched on ImageNet classification?
• RQ2 Does integrating pre-training within a one-shot NAS framework make backbone search for detectors computationally feasible?
• RQ3 What architectural patterns emerge when NAS is optimized for object detection detectors (FPN, RetinaNet) and datasets (COCO, VOC)?

Key findings

• DetNASNet achieves 40.2 mmAP on COCO with 1.3G FLOPs, outperforming ResNet-50 under the same detector (FPN).
• DetNASNet (3.8) reaches 42.0 mmAP with 3.8G FLOPs, surpassing ResNet-50 by 4.7% and ResNet-101 by 2.0%.
• Compared to a hand-crafted ShuffleNetv2-40 with identical FLOPs (1.3G), DetNASNet is better by 0.8 mmAP.
• Across detectors and datasets, networks searched for detection consistently beat networks searched on ImageNet classification by more than 3% (VOC) and 1% (COCO).
• The DetNAS framework requires about 44 GPU-days, roughly twice the cost of a standard detector training, making backbone search practical.