[论文解读] DETRs Beat YOLOs on Real-time Object Detection

Y. Zhao, Wenyu Lv|arXiv (Cornell University)|Apr 17, 2023

Advanced Neural Network Applications被引用 222

一句话总结

本文基于其实验结果表明，DETR 为基础的检测器在实时目标检测方面优于 YOLO 系列检测器。

ABSTRACT

The YOLO series has become the most popular framework for real-time object detection due to its reasonable trade-off between speed and accuracy. However, we observe that the speed and accuracy of YOLOs are negatively affected by the NMS. Recently, end-to-end Transformer-based detectors (DETRs) have provided an alternative to eliminating NMS. Nevertheless, the high computational cost limits their practicality and hinders them from fully exploiting the advantage of excluding NMS. In this paper, we propose the Real-Time DEtection TRansformer (RT-DETR), the first real-time end-to-end object detector to our best knowledge that addresses the above dilemma. We build RT-DETR in two steps, drawing on the advanced DETR: first we focus on maintaining accuracy while improving speed, followed by maintaining speed while improving accuracy. Specifically, we design an efficient hybrid encoder to expeditiously process multi-scale features by decoupling intra-scale interaction and cross-scale fusion to improve speed. Then, we propose the uncertainty-minimal query selection to provide high-quality initial queries to the decoder, thereby improving accuracy. In addition, RT-DETR supports flexible speed tuning by adjusting the number of decoder layers to adapt to various scenarios without retraining. Our RT-DETR-R50 / R101 achieves 53.1% / 54.3% AP on COCO and 108 / 74 FPS on T4 GPU, outperforming previously advanced YOLOs in both speed and accuracy. We also develop scaled RT-DETRs that outperform the lighter YOLO detectors (S and M models). Furthermore, RT-DETR-R50 outperforms DINO-R50 by 2.2% AP in accuracy and about 21 times in FPS. After pre-training with Objects365, RT-DETR-R50 / R101 achieves 55.3% / 56.2% AP. The project page: https://zhao-yian.github.io/RTDETR.

研究动机与目标

• Motivate the comparison between transformer-based DETR detectors and YOLO-series detectors for real-time object detection.
• Investigate the performance gap between DETR approaches and YOLO approaches under real-time constraints.
• Provide empirical evidence to guide model selection for real-time detection tasks.

提出的方法

• Conduct experiments comparing DETR-based detectors with YOLO-series models under real-time constraints.
• Leverage end-to-end DETR architectures as baselines for real-time performance evaluation.
• Cite and compare relevant prior works on DETR and YOLO for contextual grounding.

实验结果

研究问题

• RQ1 Do DETR-based detectors beat YOLO-series detectors in real-time object detection scenarios?
• RQ2 What are the observed strengths and limitations of DETR methods compared to YOLO methods under real-time requirements?
• RQ3 How does end-to-end DETR performance compare to fast, one-stage YOLO variants in practical deployment?

主要发现

• DETR-based detectors outperform YOLO-series detectors in real-time object detection according to the authors’ experiments.
• The findings position DETRs as competitive or superior choices for real-time detection tasks in certain settings.
• The paper provides empirical evaluation to support the DETR vs. YOLO comparison in practical deployment contexts.