[Paper Review] MDSSD: Multi-scale Deconvolutional Single Shot Detector for Small Objects

Lisha Cui, Rui Ma+5 more|arXiv (Cornell University)|May 18, 2018

Advanced Neural Network ApplicationsComputer Science32 references63 citations

TL;DR

MDSSD introduces multi-scale deconvolutional fusion blocks that upsample high-level features and fuse them with shallow layers to boost small object detection, achieving state-of-the-art results on TT100K, VOC2007, and COCO.

ABSTRACT

For most of the object detectors based on multi-scale feature maps, the shallow layers are rich in fine spatial information and thus mainly responsible for small object detection. The performance of small object detection, however, is still less than satisfactory because of the deficiency of semantic information on shallow feature maps. In this paper, we design a Multi-scale Deconvolutional Single Shot Detector (MDSSD), especially for small object detection. In MDSSD, multiple high-level feature maps at different scales are upsampled simultaneously to increase the spatial resolution. Afterwards, we implement the skip connections with low-level feature maps via Fusion Block. The fusion feature maps, named Fusion Module, are of strong feature representational power of small instances. It is noteworthy that these high-level feature maps utilized in Fusion Block preserve both strong semantic information and some fine details of small instances, rather than the top-most layer where the representation of fine details for small objects are potentially wiped out. The proposed framework achieves 77.6% mAP for small object detection on the challenging dataset TT100K with 512 x 512 input, outperforming other detectors with a large margin. Moreover, it can also achieve state-of-the-art results for general object detection on PASCAL VOC2007 test and MS COCO test-dev2015, especially achieving 2 to 5 points improvement on small object categories.

Motivation & Objective

• Motivate the challenge of small object detection and the limitations of existing multi-scale detectors.
• Develop a multi-scale deconvolutional framework to preserve spatial details while leveraging semantic richness.
• Incorporate Fusion Blocks to fuse high-level and low-level features for small object detection.
• Evaluate MDSSD on TT100K, PASCAL VOC2007, and MS COCO to demonstrate improvements over SSD and related methods.

Proposed method

• Apply deconvolution layers to high-level feature maps at different scales to upsample spatial resolution.
• Introduce Fusion Blocks that fuse upsampled high-level features with corresponding shallow features via skip connections.
• Create three Fusion Modules (Module 1, Module 2, Module 3) that operate before the deepest SSD layers (conv11_2) to recover small-object details.
• Make predictions on both new Fusion Modules and original SSD layers in parallel.
• Train with a loss that is a weighted sum of localization (Smooth L1) and confidence (Softmax) losses.

Experimental results

Research questions

• RQ1 How does feature resolution affect small object detection in SSD-like architectures?
• RQ2 Can multi-scale deconvolutional upsampling plus feature fusion improve small-object detection without sacrificing large-object performance?
• RQ3 What is the impact of adding Fusion Modules on detection accuracy across datasets (TT100K, VOC2007, COCO)?

Key findings

• MDSSD512 achieves mAP of 77.6% on TT100K, outperforming SSD512 (68.7%) and RFB Net (74.4%).
• MDSSD512 surpasses Faster R-CNN variants (52.9% and 61.1%) on TT100K despite smaller input size (512×512).
• MDSSD300 on PASCAL VOC2007 achieves 78.6% mAP, comparable to DSSD321, and with ResNet-101 backbones reaches 81.0% (MDSSD512*).
• On COCO, MDSSD300 and MDSSD512 achieve 10.8% AP and 13.9% AP for small objects (area < 32^2), respectively, higher than SSD, DSSD, and DSOD baselines.
• MDSSD also reports higher average recall (AR) for small objects, indicating improved small-object detection capability.