[Paper Review] Single-Shot Refinement Neural Network for Object Detection

Shifeng Zhang, Longyin Wen+3 more|arXiv (Cornell University)|Nov 18, 2017

Advanced Neural Network ApplicationsComputer Science46 references111 citations

TL;DR

RefineDet proposes a two-stage-like refinement inside a single-shot detector, using an Anchor Refinement Module (ARM) and an Object Detection Module (ODM) linked by Transfer Connection Blocks to achieve state-of-the-art accuracy with high efficiency.

ABSTRACT

For object detection, the two-stage approach (e.g., Faster R-CNN) has been achieving the highest accuracy, whereas the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, in this paper, we propose a novel single-shot based detector, called RefineDet, that achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. RefineDet consists of two inter-connected modules, namely, the anchor refinement module and the object detection module. Specifically, the former aims to (1) filter out negative anchors to reduce search space for the classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes the refined anchors as the input from the former to further improve the regression and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multi-task loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO demonstrate that RefineDet achieves state-of-the-art detection accuracy with high efficiency. Code is available at https://github.com/sfzhang15/RefineDet

Motivation & Objective

• Motivate bridging the accuracy of two-stage detectors with the speed of one-stage detectors.
• Introduce a two-module single-shot framework (ARM and ODM) that sequentially refines anchors and then detects objects.
• Address class imbalance and small-object accuracy through negative anchor filtering and two-step regression.
• Demonstrate end-to-end trainability and state-of-the-art performance on PASCAL VOC 2007/2012 and MS COCO.

Proposed method

• Propose RefineDet consisting of an Anchor Refinement Module (ARM) and an Object Detection Module (ODM) connected via Transfer Connection Blocks (TCB).
• ARM filters negative anchors and coarse-refines anchor locations/sizes to better initialize the regressor.
• ODM uses refined anchors from ARM to perform accurate classification and precise localization.
• TCBs transfer ARM features to ODM and incorporate large-scale context for improved predictions.
• Train the whole network end-to-end with a multi-task loss combining ARM and ODM objectives.
• Use a two-step cascaded regression: ARM first refines anchors, then ODM predicts final boxes and class labels.

Experimental results

Research questions

• RQ1 Can a two-module, single-shot detector match or exceed two-stage methods in accuracy while maintaining one-stage efficiency?
• RQ2 Does anchor refinement plus a transfer connection block improve localization and classification over standard one-shot detectors?
• RQ3 What is the impact of negative anchor filtering and two-step cascaded regression on detection performance?
• RQ4 Can end-to-end training of ARM and ODM yield state-of-the-art results on VOC and MS COCO across multiple backbones?

Key findings

• RefineDet achieves state-of-the-art or competitive mAP on VOC 2007/2012 and MS COCO across backbones (VGG-16, ResNet-101).
• Two-step cascaded regression (ARM refinement followed by ODM) significantly improves accuracy versus one-step regression alone.
• Negative Anchor Filtering reduces training/inference noise and improves mAP (ablation shows 0.5% absolute gain on VOC with filtering).
• Transfer Connection Blocks (TCB) enable effective feature transfer from ARM to ODM and incorporate context, yielding up to 1.1% mAP improvement in ablation.
• RefineDet320/512 with multi-scale testing (320+, 512+) achieve top VOC2012 and VOC2007 results, and RefineDet+ variants reach state-of-the-art AP on MS COCO with 320/512 inputs.