[Paper Review] EfficientDet: Scalable and Efficient Object Detection

Mingxing Tan, Ruoming Pang+1 more|arXiv (Cornell University)|Nov 20, 2019

Advanced Neural Network Applications41 references521 citations

TL;DR

EfficientDet introduces a weighted bidirectional feature pyramid network (BiFPN) and a compound scaling method to create a family of detectors that achieve state-of-the-art COCO accuracy with far fewer parameters and FLOPs.

ABSTRACT

Model efficiency has become increasingly important in computer vision. In this paper, we systematically study neural network architecture design choices for object detection and propose several key optimizations to improve efficiency. First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows easy and fast multiscale feature fusion; Second, we propose a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time. Based on these optimizations and better backbones, we have developed a new family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints. In particular, with single model and single-scale, our EfficientDet-D7 achieves state-of-the-art 55.1 AP on COCO test-dev with 77M parameters and 410B FLOPs, being 4x - 9x smaller and using 13x - 42x fewer FLOPs than previous detectors. Code is available at https://github.com/google/automl/tree/master/efficientdet.

Motivation & Objective

• Investigate design choices for efficient one-stage object detectors under real-world resource constraints.
• Develop a scalable architecture that improves both accuracy and efficiency across a wide range of FLOPs and parameters.
• Propose a learnable feature fusion mechanism and a unified scaling strategy to balance backbone, feature network, and prediction heads.

Proposed method

• Propose a weighted bi-directional feature pyramid network (BiFPN) with cross-scale connections for efficient multi-scale feature fusion.
• Introduce fast normalized fusion to replace softmax-weighted fusion for lower latency.
• Adopt EfficientNet backbones and a shared class/box head within a single architecture (EfficientDet).
• Develop a compound scaling method that jointly scales backbone width/depth, BiFPN depth/width, and box/class prediction networks, guided by a simple coefficient φ.
• Repeat BiFPN layers and prediction heads to match different resource budgets (D0–D7 configurations).
• Evaluate using COCO detection and Pascal VOC segmentation to demonstrate efficiency and accuracy gains.

Experimental results

Research questions

• RQ1 Can a learnable, efficient cross-scale feature fusion (BiFPN) outperform existing FPN variants in both accuracy and computational cost?
• RQ2 Does a principled, compound scaling strategy across backbone, feature network, and prediction heads yield superior accuracy-efficiency trade-offs across multiple resource constraints?
• RQ3 How do backbones like EfficientNet and a shared class/box head contribute to overall detector efficiency?
• RQ4 What is the impact of weighted vs. unweighted feature fusion on performance and speed?

Key findings

• EfficientDet-D7x achieves 55.1 AP on COCO test-dev with 77M parameters and 410B FLOPs, representing superior accuracy with substantially fewer computations and parameters compared to prior detectors.
• BiFPN with weighted feature fusion achieves better accuracy while using fewer parameters and FLOPs than other cross-scale networks (e.g., NAS-FPN, PANet).
• Fast normalized fusion delivers similar accuracy to softmax-based fusion but runs up to ~30% faster on GPUs, improving end-to-end efficiency.
• Compound scaling across backbone, BiFPN, and prediction heads yields better accuracy-efficiency trade-offs than single-dimension scaling methods; D0–D7 configurations cover a wide range of resource budgets.
• EfficientNet backbones combined with BiFPN and shared box/class heads substantially reduce model size and FLOPs while maintaining or improving accuracy across COCO and VOC tasks.