[Paper Review] Comparative Analysis of YOLOv9, YOLOv10 and RT-DETR for Real-Time Weed Detection

Ahmet Oğuz Saltık, Alicia Allmendinger+1 more|arXiv (Cornell University)|Dec 18, 2024

Smart Agriculture and AIAgricultural and Biological Sciences3 citations

TL;DR

This study evaluates YOLOv9, YOLOv10, and RT-DETR for real-time weed detection in agriculture, comparing mAP and inference time across multiple model sizes and image resolutions (320–960px) on RTX3080, RTX3090, and RTX4090 GPUs. YOLOv9 and YOLOv10 achieve the best balance of accuracy and speed, while RT-DETR (l) delivers higher mAP at the cost of significantly longer inference times, especially at larger resolutions.

ABSTRACT

This paper presents a comprehensive evaluation of state-of-the-art object detection models, including YOLOv9, YOLOv10, and RT-DETR, for the task of weed detection in smart-spraying applications focusing on three classes: Sugarbeet, Monocot, and Dicot. The performance of these models is compared based on mean Average Precision (mAP) scores and inference times on different GPU and CPU devices. We consider various model variations, such as nano, small, medium, large alongside different image resolutions (320px, 480px, 640px, 800px, 960px). The results highlight the trade-offs between inference time and detection accuracy, providing valuable insights for selecting the most suitable model for real-time weed detection. This study aims to guide the development of efficient and effective smart spraying systems, enhancing agricultural productivity through precise weed management.

Motivation & Objective

• To evaluate the performance of YOLOv9, YOLOv10, and RT-DETR for real-time weed detection in smart-spraying applications.
• To analyze the trade-off between detection accuracy (mAP) and inference speed across different model architectures and sizes.
• To assess the impact of input image resolution (320px to 960px) on model accuracy and inference time.
• To identify the most suitable model for real-time agricultural applications based on hardware constraints and performance metrics.
• To guide the selection of object detection models in precision agriculture by providing empirical benchmarks across diverse hardware and resolution settings.

Proposed method

• The study evaluates YOLOv9 and YOLOv10 in nano, small, medium, and large variants, and RT-DETR (l) as a two-stage transformer-based detector.
• Models are tested on image resolutions of 320px, 480px, 640px, 800px, and 960px to assess resolution-dependent performance.
• Performance is measured using mAP50 and mAP50-95 as accuracy metrics, with inference time recorded on three GPU devices: RTX3080 laptop, RTX3090, and RTX4090.
• Evaluation is conducted on a dataset with three weed classes: Sugarbeet, Monocot, and Dicot, representative of real-world smart-spraying scenarios.
• Time distribution analysis is performed to understand computational bottlenecks in each model architecture.
• Comparative visualizations and statistical analysis are used to assess detection quality and model robustness across different configurations.

Experimental results

Research questions

• RQ1 How do YOLOv9, YOLOv10, and RT-DETR (l) compare in terms of mAP50 and mAP50-95 across varying image resolutions?
• RQ2 What is the trade-off between inference time and detection accuracy for each model across different GPU devices?
• RQ3 How does increasing image resolution affect the mAP and inference time of each model?
• RQ4 Which model offers the best balance of speed and accuracy for real-time weed detection in agricultural settings?
• RQ5 At what image resolution does a significant drop in accuracy occur, and how do model variants respond to this threshold?

Key findings

• Larger image resolutions (e.g., 960px) consistently improve mAP scores for all models, with RT-DETR (l) showing the most significant gains at 960px (mAP50: 68.1, mAP50-95: 87.2).
• YOLOv9 (c) and YOLOv10 (l) achieve high mAP values (mAP50: ~85–87) with inference times under 30ms on RTX3080, indicating strong real-time suitability.
• RT-DETR (l) achieves the highest mAP50-95 score of 87.2 at 960px resolution but has the longest inference time (34.7ms on RTX3080), indicating a high computational cost.
• The performance gap in accuracy between 640px and 960px is minimal, but a significant drop in accuracy occurs below 640px, especially for RT-DETR (l).
• YOLOv9 and YOLOv10 show stable performance across different image sizes and GPUs, with minimal variation in mAP and inference time, indicating robustness.
• Time distribution analysis reveals that RT-DETR (l) spends more time on attention mechanisms and feature processing, contributing to its slower inference compared to one-stage YOLO models.