[Paper Review] DAMO-YOLO : A Report on Real-Time Object Detection Design

Xianzhe Xu, Yiqi Jiang+4 more|arXiv (Cornell University)|Nov 23, 2022

Advanced Neural Network Applications118 citations

TL;DR

DAMO-YOLO extends YOLO with NAS-backed backbones, an efficient RepGFPN neck, a compact ZeroHead, AlignOTA label assignment, and distillation to achieve state-of-the-art real-time object detection performance on COCO across general and lightweight models.

ABSTRACT

In this report, we present a fast and accurate object detection method dubbed DAMO-YOLO, which achieves higher performance than the state-of-the-art YOLO series. DAMO-YOLO is extended from YOLO with some new technologies, including Neural Architecture Search (NAS), efficient Reparameterized Generalized-FPN (RepGFPN), a lightweight head with AlignedOTA label assignment, and distillation enhancement. In particular, we use MAE-NAS, a method guided by the principle of maximum entropy, to search our detection backbone under the constraints of low latency and high performance, producing ResNet/CSP-like structures with spatial pyramid pooling and focus modules. In the design of necks and heads, we follow the rule of ``large neck, small head''.We import Generalized-FPN with accelerated queen-fusion to build the detector neck and upgrade its CSPNet with efficient layer aggregation networks (ELAN) and reparameterization. Then we investigate how detector head size affects detection performance and find that a heavy neck with only one task projection layer would yield better results.In addition, AlignedOTA is proposed to solve the misalignment problem in label assignment. And a distillation schema is introduced to improve performance to a higher level. Based on these new techs, we build a suite of models at various scales to meet the needs of different scenarios. For general industry requirements, we propose DAMO-YOLO-T/S/M/L. They can achieve 43.6/47.7/50.2/51.9 mAPs on COCO with the latency of 2.78/3.83/5.62/7.95 ms on T4 GPUs respectively. Additionally, for edge devices with limited computing power, we have also proposed DAMO-YOLO-Ns/Nm/Nl lightweight models. They can achieve 32.3/38.2/40.5 mAPs on COCO with the latency of 4.08/5.05/6.69 ms on X86-CPU. Our proposed general and lightweight models have outperformed other YOLO series models in their respective application scenarios.

Motivation & Objective

• Motivate real-time object detection improvements for industry deployment.
• Develop a YOLO-based detector with latency-aware neural architecture search.
• Design an efficient neck and lightweight head to optimize latency-accuracy trade-offs.
• Improve label assignment to align classification and regression in dynamic settings.
• Incorporate distillation to boost performance of smaller models.

Proposed method

• Use MAE-NAS to search latency-constrained backbones producing ResNet-like/CSP-like structures.
• Develop Efficient RepGFPN neck with flexible channel dimensions across scales and remove upsampling in queen-fusion.
• Introduce ZeroHead to keep only a task projection layer, enabling large neck with a small head.
• Propose AlignOTA, a aligned dynamic label assignment balancing classification and regression costs.
• Apply distillation with two-stage training and Channel-wise Dynamic Temperature for improved small-model performance.

Experimental results

Research questions

• RQ1 How can NAS-designed backbones under latency constraints improve COCO mAP for real-time detectors?
• RQ2 What neck/head configuration yields best accuracy under real-time constraints?
• RQ3 Does AlignOTA improve alignment between classification and regression in dynamic label assignment for YOLO-like detectors?
• RQ4 How can distillation boost performance of small DAMO-YOLO models without harming real-time throughput?

Key findings

• DAMO-YOLO-T/S/M/L achieve 43.6/47.7/50.2/51.9 mAPs on COCO with 2.78/3.83/5.62/7.95 ms latency on T4 GPUs.
• DAMO-YOLO-Ns/Nm/Nl lightweight models achieve 32.3/38.2/40.5 mAPs on COCO with 4.08/5.05/6.69 ms latency on X86-CPU.
• MAE-NAS backbones (MAE-Res, MAE-CSP) outperform CSP-Darknet baselines in accuracy/latency trade-offs, especially on larger/deeper networks.
• A large-neck/one-projection-head (ZeroHead) design yields strong performance with reduced computation.
• AlignOTA improves label assignment over ATSS/sOTA/TOOD, achieving higher AP.
• Distillation (CWD favored) boosts performance for small to medium DAMO-YOLO models.