[Paper Review] DSOD: Learning Deeply Supervised Object Detectors from Scratch

Zhiqiang Shen, Zhuang Liu+4 more|arXiv (Cornell University)|Aug 3, 2017

Advanced Neural Network Applications30 references96 citations

TL;DR

DSOD trains object detectors from scratch within a proposal-free, densely connected framework inspired by DenseNets and SSD, achieving state-of-the-art results with smaller models and real-time speed.

ABSTRACT

We present Deeply Supervised Object Detector (DSOD), a framework that can learn object detectors from scratch. State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks. Model fine-tuning for the detection task could alleviate this bias to some extent but not fundamentally. Besides, transferring pre-trained models from classification to detection between discrepant domains is even more difficult (e.g. RGB to depth images). A better solution to tackle these two critical problems is to train object detectors from scratch, which motivates our proposed DSOD. Previous efforts in this direction mostly failed due to much more complicated loss functions and limited training data in object detection. In DSOD, we contribute a set of design principles for training object detectors from scratch. One of the key findings is that deep supervision, enabled by dense layer-wise connections, plays a critical role in learning a good detector. Combining with several other principles, we develop DSOD following the single-shot detection (SSD) framework. Experiments on PASCAL VOC 2007, 2012 and MS COCO datasets demonstrate that DSOD can achieve better results than the state-of-the-art solutions with much more compact models. For instance, DSOD outperforms SSD on all three benchmarks with real-time detection speed, while requires only 1/2 parameters to SSD and 1/10 parameters to Faster RCNN. Our code and models are available at: https://github.com/szq0214/DSOD .

Motivation & Objective

• Motivate training object detectors from scratch to avoid biases from pre-trained classification models.
• Propose design principles for resource-efficient, high-accuracy detectors.
• Develop a DSOD framework built on a proposal-free, single-shot detection paradigm with deep supervision.
• Demonstrate that DSOD achieves state-of-the-art results on VOC 2007, VOC 2012, and MS COCO with smaller models.

Proposed method

• Adopt a proposal-free, single-shot detection framework based on SSD for speed.
• Introduce deep supervision via dense layer-wise connections, enabling implicit auxiliary supervision.
• Incorporate a stem block to reduce information loss from raw inputs.
• Use a dense prediction structure that fuses multi-scale feature maps for each prediction scale.
• Include a transition w/o pooling layer to increase dense blocks without downsampling.
• Train all networks from scratch on standard detection benchmarks.

Experimental results

Research questions

• RQ1 Can object detectors be effectively trained from scratch without pre-trained classification models?
• RQ2 What network design principles enable high accuracy and efficiency for scratch-trained detectors?
• RQ3 How do dense, multi-scale prediction structures affect accuracy and parameter efficiency in scratch-trained detectors?

Key findings

• DSOD achieves competitive and sometimes superior mAP on VOC 2007, VOC 2012, and MS COCO without ImageNet pre-training.
• DSOD300 with plain connections reaches 77.3% mAP on VOC 2007 test when trained on 07+12; using dense prediction raises to 77.7%.
• With COCO data (07+12+COCO), DSOD300 with dense prediction reaches 81.7% mAP on VOC 2007 test.
• DSOD offers real-time detection speed (e.g., 20.6 fps at 300x300 on Titan X with plain structure) and uses far fewer parameters than SSD/Faster R-CNN baselines.
• A stem block and transition w/o pooling layer significantly improve accuracy, while dense prediction structures reduce parameters and can improve accuracy.
• DSOD trained from scratch can rival or surpass models fine-tuned from pre-trained classifiers, highlighting the value of architecture design for detection without pre-training.