[Paper Review] Pelee: A Real-Time Object Detection System on Mobile Devices

Robert J. Wang, Xiang Li+1 more|arXiv (Cornell University)|Apr 18, 2018

Advanced Neural Network ApplicationsComputer Science19 references101 citations

TL;DR

The paper proposes PeleeNet and a real-time SSD-based detection system (Pelee) built with conventional convolution, achieving high accuracy and real-time speeds on mobile hardware, outperforming several mobile detectors in both accuracy and efficiency.

ABSTRACT

An increasing need of running Convolutional Neural Network (CNN) models on mobile devices with limited computing power and memory resource encourages studies on efficient model design. A number of efficient architectures have been proposed in recent years, for example, MobileNet, ShuffleNet, and MobileNetV2. However, all these models are heavily dependent on depthwise separable convolution which lacks efficient implementation in most deep learning frameworks. In this study, we propose an efficient architecture named PeleeNet, which is built with conventional convolution instead. On ImageNet ILSVRC 2012 dataset, our proposed PeleeNet achieves a higher accuracy and over 1.8 times faster speed than MobileNet and MobileNetV2 on NVIDIA TX2. Meanwhile, PeleeNet is only 66% of the model size of MobileNet. We then propose a real-time object detection system by combining PeleeNet with Single Shot MultiBox Detector (SSD) method and optimizing the architecture for fast speed. Our proposed detection system2, named Pelee, achieves 76.4% mAP (mean average precision) on PASCAL VOC2007 and 22.4 mAP on MS COCO dataset at the speed of 23.6 FPS on iPhone 8 and 125 FPS on NVIDIA TX2. The result on COCO outperforms YOLOv2 in consideration of a higher precision, 13.6 times lower computational cost and 11.3 times smaller model size.

Motivation & Objective

• Motivate real-time CNNs on devices with limited compute and memory.
• Design an efficient CNN (PeleeNet) using conventional convolutions without depthwise separable convolutions.
• Integrate PeleeNet with an optimized SSD-based detector (Pelee) for fast object detection on mobile hardware.
• Evaluate speed-accuracy trade-offs on mobile devices and compare with state-of-the-art detectors.

Proposed method

• Introduce PeleeNet, a DenseNet-inspired network with 2-way dense layers, stem block, dynamic bottleneck channels, and post-activation for faster inference.
• Adopt an SSD-based detector with 5 feature-map scales (19x19, 10x10, 5x5, 3x3, 1x1) and residual prediction blocks preceding predictions.
• Use 1x1 convolution kernels for prediction to reduce FLOPs and model size while maintaining accuracy.
• Fine-tune and train PeleeNet on ImageNet and evaluate on VOC2007 and COCO datasets; optimize for FP16 on TX2 and CoreML on iPhone 8.
• Compare against MobileNet, ShuffleNet, YOLOv2, and SSD variants to demonstrate speed-accuracy gains.

Experimental results

Research questions

• RQ1 Can a convolutional network built with conventional convolutions (not depthwise separable) achieve competitive accuracy with smaller model size for mobile vision tasks?
• RQ2 Does integrating PeleeNet with an SSD-based detector and specific design choices yield real-time inference on mobile devices without sacrificing accuracy?
• RQ3 How do design choices like 5-feature-map scales, residual prediction blocks, and 1x1 kernels impact accuracy and speed on embedded hardware?
• RQ4 How does Pelee fare against established detectors (e.g., YOLOv2, SSD-MobileNet) on VOC and COCO benchmarks in terms of mAP and computational cost?

Key findings

• PeleeNet achieves 72.6% top-1 accuracy on ImageNet ILSVRC 2012 with 508 MFLOPs and 2.8M parameters, outperforming MobileNet and ShuffleNet at similar or lower model sizes.
• Pelee, the SSD-based detector using PeleeNet features, reaches 76.4% mAP on VOC07 and 22.4 mAP on COCO, with substantially lower model size and FLOPs than competing detectors.
• On real devices, Pelee runs at 23.6 FPS on iPhone 8 and 125 FPS on NVIDIA TX2 in FP16, beating SSD+MobileNet variants in speed and often in accuracy.
• On COCO test-dev2015, Pelee achieves higher mAP than SSD+MobileNet and YOLOv2, with 3.7x faster speed and 11.3x smaller model size than YOLOv2.
• Compared to YOLOv2, Pelee shows higher precision and significantly lower computational cost and model size for COCO.
• The design choices of residual prediction blocks and 1x1 kernels contribute to reduced FLOPs and parameter counts while maintaining competitive accuracy.