[Paper Review] YOLOv4: Optimal Speed and Accuracy of Object Detection

Alexey Bochkovskiy, Chien-Yao Wang+1 more|arXiv (Cornell University)|Apr 23, 2020

Advanced Neural Network Applications101 references10,401 citations

TL;DR

This paper presents YOLOv4, a real-time object detector that achieves state-of-the-art accuracy with real-time speed on a single GPU by combining new features (Bag of Freebies and Bag of Specials) and an optimized backbone (CSPDarknet53) with SPP and PANet neck, validated on MS COCO.

ABSTRACT

There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at https://github.com/AlexeyAB/darknet

Motivation & Objective

• Develop a fast, real-time object detector operable on a conventional GPU
• Investigate the impact of state-of-the-art training and architectural features on detector accuracy
• Design a backbone/neck/head combination optimized for single-GPU training and inference
• Assess and compare the proposed model against contemporary detectors in speed/accuracy
• Provide reproducible settings enabling training on standard GPUs without SyncBN or multi-GPU setups.

Proposed method

• Adopt CSPDarknet53 backbone with SPP and PANet neck and a YOLOv3-like head as the YOLOv4 architecture.
• Utilize Bag of Freebies (BoF) including Mosaic data augmentation, CutMix, label smoothing, DropBlock, and Mish activation.
• Employ Bag of Specials (BoS) such as Mish activation, CSP, and MiWRC to boost performance with minimal inference cost.
• Apply CIoU loss, CmBN, DropBlock, Mosaic/SAT data augmentation, multi-anchor training, cosine annealing, and optimized hyperparameters.
• Conduct extensive ablation studies to evaluate BoF/BoS components on classifier and detector performance on ImageNet and MS COCO.
• Demonstrate real-time performance (≈65 FPS) on Tesla V100 with 43.5% AP on COCO using 608x608 input.

Experimental results

Research questions

• RQ1 What combination of backbone, neck, and head yields the best speed-accuracy trade-off for real-time detection on a single GPU?
• RQ2 How do Bag of Freebies and Bag of Specials improvements impact detector accuracy without increasing inference cost?
• RQ3 Can single-GPU training with modified normalization and data augmentation achieve state-of-the-art COCO results?
• RQ4 How does YOLOv4 compare to contemporary detectors in speed and accuracy across common GPUs?
• RQ5 What are the optimal training hyperparameters and data augmentation strategies for a fast, accurate detector?

Key findings

• YOLOv4 achieves 43.5% AP (65.7% AP50) for MS COCO at ~65 FPS on a Tesla V100.
• The CSPDarknet53 backbone with SPP and PANet neck outperforms alternatives in COCO object detection while enabling real-time speed.
• BoF methods (CutMix, Mosaic, label smoothing, DropBlock, Mish activation) improve classifier/ detector performance without increasing inference cost.
• BoS components (Mish, SPP, SAM, PAN, DIoU-NMS) contribute to higher AP with manageable FPS, with CSPDarknet53-based detectors showing strong results.
• On 416, 512, and 608 input sizes, YOLOv4 achieves progressively higher AP values with corresponding FPS: e.g., 416: AP 41.2%, FPS 38; 512: AP 43.0%, FPS 31; 608: AP 43.5%, FPS 23.
• The model operates on a single GPU without SyncBN, enabling broader reproducibility.