[Paper Review] PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices

Guanghua Yu, Qinyao Chang+13 more|arXiv (Cornell University)|Nov 1, 2021

Advanced Neural Network Applications89 citations

TL;DR

PP-PicoDet proposes a lightweight anchor-free object detector optimized for mobile devices, using Enhanced ShuffleNet (ESNet) backbones, CSP-PAN neck, and SimOTA-based dynamic label assignment to achieve strong accuracy with low latency on ARM CPUs. It reports state-of-the-art results among lightweight detectors.

ABSTRACT

The better accuracy and efficiency trade-off has been a challenging problem in object detection. In this work, we are dedicated to studying key optimizations and neural network architecture choices for object detection to improve accuracy and efficiency. We investigate the applicability of the anchor-free strategy on lightweight object detection models. We enhance the backbone structure and design the lightweight structure of the neck, which improves the feature extraction ability of the network. We improve label assignment strategy and loss function to make training more stable and efficient. Through these optimizations, we create a new family of real-time object detectors, named PP-PicoDet, which achieves superior performance on object detection for mobile devices. Our models achieve better trade-offs between accuracy and latency compared to other popular models. PicoDet-S with only 0.99M parameters achieves 30.6% mAP, which is an absolute 4.8% improvement in mAP while reducing mobile CPU inference latency by 55% compared to YOLOX-Nano, and is an absolute 7.1% improvement in mAP compared to NanoDet. It reaches 123 FPS (150 FPS using Paddle Lite) on mobile ARM CPU when the input size is 320. PicoDet-L with only 3.3M parameters achieves 40.9% mAP, which is an absolute 3.7% improvement in mAP and 44% faster than YOLOv5s. As shown in Figure 1, our models far outperform the state-of-the-art results for lightweight object detection. Code and pre-trained models are available at https://github.com/PaddlePaddle/PaddleDetection.

Motivation & Objective

• Investigate anchor-free, lightweight object detectors suitable for mobile devices.
• Improve backbone and neck architectures to boost feature extraction with low parameter count.
• Enhance label assignment and loss functions for stable, efficient training.
• Explore NAS-assisted backbone design to optimize accuracy-latency trade-offs for detection on mobile hardware.

Proposed method

• Adopt CSP-PAN neck with 1x1 channel equalization to reduce parameters and enlarge receptive field with 5x5 depthwise separable convolutions.
• Use Enhanced ShuffleNet (ESNet) as backbone with SE and Ghost modules to improve representation while staying lightweight.
• Employ SimOTA dynamic label assignment with a cost matrix based on Varifocal Loss and GIoU loss to improve positive sample selection and localization.
• Train a one-shot detection supernet and perform EA-based architecture search focused on channel ratios for backbones, then retrain selected architectures on COCO.
• Replace activations with H-Swish for mobile efficiency and apply cosine learning rate decay with modest data augmentation.

Experimental results

Research questions

• RQ1 Can anchor-free detectors be optimized for mobile devices without sacrificing accuracy?
• RQ2 Do backbone and neck architectural enhancements (ESNet and CSP-PAN) yield meaningful gains in mAP and latency on ARM CPUs?
• RQ3 Does dynamic SimOTA label assignment with a Varifocal/GIoU cost matrix improve training stability and detection quality for lightweight models?
• RQ4 Can one-shot NAS tuned for detection tasks find architectures that improve latency-accuracy trade-offs on mobile hardware?

Key findings

• PP-PicoDet-S (0.99M params) achieves 30.6% mAP (0.5:0.95) with 1.08G FLOPs and 123 FPS on mobile ARM CPU (320 input), outperforming YOLOX-Nano and NanoDet at similar scales.
• PP-PicoDet-L (3.3M params) reaches 40.9% mAP (0.5:0.95) with 8.74G FLOPs and is ~44% faster than YOLOv5s under comparable conditions.
• A lighter CSP-PAN neck and depthwise separable convolutions yield substantial accuracy gains with minimal parameter overhead compared to baselines.
• ESNet backbone demonstrates better accuracy and lower latency than ShuffleNetV2-1.5x in ImageNet-1k, supporting the efficiency gains for detection tasks.
• NAS-driven backbone optimization (ESNet-0.75x) reduces parameters with negligible mAP loss while significantly reducing latency.