[Paper Review] Enhancement of SSD by concatenating feature maps for object detection

Jisoo Jeong, Hyojin Park+1 more|arXiv (Cornell University)|May 26, 2017

Advanced Neural Network Applications50 citations

TL;DR

Rainbow SSD (R-SSD) enhances SSD by concatenating features across scales to exploit layer relationships, yielding higher mAP with competitive speed without changing the base network.

ABSTRACT

We propose an object detection method that improves the accuracy of the conventional SSD (Single Shot Multibox Detector), which is one of the top object detection algorithms in both aspects of accuracy and speed. The performance of a deep network is known to be improved as the number of feature maps increases. However, it is difficult to improve the performance by simply raising the number of feature maps. In this paper, we propose and analyze how to use feature maps effectively to improve the performance of the conventional SSD. The enhanced performance was obtained by changing the structure close to the classifier network, rather than growing layers close to the input data, e.g., by replacing VGGNet with ResNet. The proposed network is suitable for sharing the weights in the classifier networks, by which property, the training can be faster with better generalization power. For the Pascal VOC 2007 test set trained with VOC 2007 and VOC 2012 training sets, the proposed network with the input size of 300 x 300 achieved 78.5% mAP (mean average precision) at the speed of 35.0 FPS (frame per second), while the network with a 512 x 512 sized input achieved 80.8% mAP at 16.6 FPS using Nvidia Titan X GPU. The proposed network shows state-of-the-art mAP, which is better than those of the conventional SSD, YOLO, Faster-RCNN and RFCN. Also, it is faster than Faster-RCNN and RFCN.

Motivation & Objective

• Motivate improving SSD accuracy without altering the base network near input data.
• Explore how inter-layer relationships in the feature pyramid can be exploited for better detection.
• Increase the number of feature maps efficiently to boost small-object detection.
• Enable weight sharing in the classifier to improve training speed and generalization.

Proposed method

• Introduce rainbow feature concatenation that combines lower-layer pooled features and upper-layer deconvolved features.
• Apply batch normalization before concatenation to normalize feature scales.
• Implement pooling-based and deconvolution-based concatenation to fuse multi-scale information.
• Propose a single classifier network with shared weights across scales by equalizing feature map channels across layers.
• Compare conventional SSD, pooling/deconvolution variants, ISSD, and R-SSD on VOC2007+2012 with 300x300 and 512x512 inputs.

Experimental results

Research questions

• RQ1 Can explicit cross-layer interactions in the SSD feature pyramid improve detection accuracy, especially for small objects?
• RQ2 Does increasing feature-map channels only in the pyramid (not in the base network) yield better accuracy with manageable speed?
• RQ3 Does a single shared classifier with matched feature-channel dimensions across scales improve generalization and training efficiency?
• RQ4 How does Rainbow concatenation affect recall and precision across object sizes on PASCAL VOC07/12?

Key findings

• R-SSD with 300 input achieves 78.5% mAP at 35.0 FPS on VOC07+12.
• R-SSD with 512 input achieves 80.8% mAP at 16.6 FPS on VOC07+12.
• R-SSD outperforms conventional SSD, YOLO, Faster-RCNN, and RFCN in mAP while maintaining competitive speed.
• A single classifier with shared weights across scales can maintain competitive accuracy and improve training efficiency on imbalanced or small datasets.
• Compared to ISSD, R-SSD offers higher accuracy with less speed degradation, and is faster than some alternatives with similar accuracy (e.g., Faster-RCNN, RFCN) at comparable mAP levels.