[Paper Review] Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism

Chengcheng Wang, Wei He+5 more|arXiv (Cornell University)|Sep 20, 2023

Advanced Neural Network Applications251 citations

TL;DR

Gold-YOLO introduces a Gather-and-Distribute (GD) neck to enhance multi-scale feature fusion in YOLO while enabling MAE-style pretraining, achieving higher AP with competitive speed.

ABSTRACT

In the past years, YOLO-series models have emerged as the leading approaches in the area of real-time object detection. Many studies pushed up the baseline to a higher level by modifying the architecture, augmenting data and designing new losses. However, we find previous models still suffer from information fusion problem, although Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) have alleviated this. Therefore, this study provides an advanced Gatherand-Distribute mechanism (GD) mechanism, which is realized with convolution and self-attention operations. This new designed model named as Gold-YOLO, which boosts the multi-scale feature fusion capabilities and achieves an ideal balance between latency and accuracy across all model scales. Additionally, we implement MAE-style pretraining in the YOLO-series for the first time, allowing YOLOseries models could be to benefit from unsupervised pretraining. Gold-YOLO-N attains an outstanding 39.9% AP on the COCO val2017 datasets and 1030 FPS on a T4 GPU, which outperforms the previous SOTA model YOLOv6-3.0-N with similar FPS by +2.4%. The PyTorch code is available at https://github.com/huawei-noah/Efficient-Computing/tree/master/Detection/Gold-YOLO, and the MindSpore code is available at https://gitee.com/mindspore/models/tree/master/research/cv/Gold_YOLO.

Motivation & Objective

• Motivate improving information fusion in YOLO necks beyond standard FPN/PANet structures.
• Propose a GD-based neck with two branches (Low-GD and High-GD) and an adjacent-layer fusion module to enhance cross-scale feature interaction.
• Enable MAE-style pretraining for YOLO-series to improve convergence and accuracy.
• Demonstrate state-of-the-art accuracy-speed trade-offs on COCO and show generality to other detection tasks.

Proposed method

• Propose Gather-and-Distribute (GD) neck that globally gathers features from all levels via Feature Alignment Module (FAM) and Information Fusion Module (IFM) and then distributes via Information Injection Module (Inject).
• Implement two GD branches: Low-GD for high-resolution, small-object features and High-GD for coarser, large-object features; use convolution-based blocks and transformer-based fusion respectively.
• Introduce a lightweight adjacent-layer fusion (LAF) module to further blend neighboring levels and balance speed-accuracy.
• Adopt MAE-style masked image modeling pretraining for backbone to improve convergence and accuracy of Gold-YOLO models.
• Replace LayerNorm with BatchNorm and GELU with ReLU in transformer blocks to accelerate inference, and adopt a hierarchical sparse pretraining approach for convnets when using MIM.

Experimental results

Research questions

• RQ1 How can a unified gather-and-distribute mechanism improve cross-level feature fusion in YOLO-neck architectures?
• RQ2 What is the impact of Low-GD and High-GD branches on small- vs large-object detection performance?
• RQ3 Does MAE-style masked image modeling pretraining improve YOLO-series performance and convergence speed?
• RQ4 How does the GD neck affect inference speed and overall AP on COCO val2017 compared to existing YOLO models?

Key findings

• Gold-YOLO-N achieves 39.9% AP on COCO val2017 with 1030 FPS on a T4 GPU (FP16, bs=1) and 1.7 ms latency (bs=1).
• Gold-YOLO-S achieves 46.1% AP (self-distillation variant 46.4% AP) with 446 FPS (bs=32) and 3.3 ms latency (bs=1).
• Gold-YOLO-M achieves 50.9% AP (68.2% AP50) with 220 FPS (bs=32) and 6.4 ms latency (bs=1).
• Gold-YOLO-L achieves 53.2% AP (70.5% AP50) with 116 FPS (bs=32) and 11.1 ms latency (bs=1).
• Gold-YOLO variants consistently outperform corresponding YOLOv6/YOLOv8/YOLOv7 baselines in AP at similar speeds; MAE-style pretraining (MIM) provides additional gains for larger models.
• Ablation shows GD structure (Low-GD+High-GD+LAF) yields higher AP and targeted improvements on small/large objects while maintaining competitive FPS.