[Paper Review] ODverse33: Is the New YOLO Version Always Better? A Multi Domain benchmark from YOLO v5 to v11

Tianyou Jiang, Yang Zhong|ArXiv.org|Feb 20, 2025

Scientific Computing and Data Management5 citations

TL;DR

The paper compiles a cross-version, multi-domain benchmark (ODverse33) comparing YOLOv5 through YOLOv11 to assess whether newer versions consistently outperform predecessors across diverse domains.

ABSTRACT

You Look Only Once (YOLO) models have been widely used for building real-time object detectors across various domains. With the increasing frequency of new YOLO versions being released, key questions arise. Are the newer versions always better than their previous versions? What are the core innovations in each YOLO version and how do these changes translate into real-world performance gains? In this paper, we summarize the key innovations from YOLOv1 to YOLOv11, introduce a comprehensive benchmark called ODverse33, which includes 33 datasets spanning 11 diverse domains (Autonomous driving, Agricultural, Underwater, Medical, Videogame, Industrial, Aerial, Wildlife, Retail, Microscopic, and Security), and explore the practical impact of model improvements in real-world, multi-domain applications through extensive experimental results. We hope this study can provide some guidance to the extensive users of object detection models and give some references for future real-time object detector development.

Motivation & Objective

• Motivate the question of whether newer YOLO versions consistently outperform older ones across diverse real-world domains.
• Introduce ODverse33 as a comprehensive, multi-domain benchmark for YOLO models from v5 to v11.
• Provide an experimental framework and datasets to analyze practical performance gains of YOLO innovations.
• Offer guidance for researchers and practitioners on choosing YOLO versions for real-time object detection tasks.

Proposed method

• Review and summarize key innovations introduced from YOLOv1 to YOLOv11.
• Construct a benchmark named ODverse33 with 33 datasets spanning 11 domains.
• Evaluate model-to-domain performance to understand real-world impact of YOLO improvements.
• Conduct extensive experiments to compare YOLO v5 through v11 across multiple domains.
• Analyze practical implications of architectural changes on detection effectiveness in real-time systems.

Experimental results

Research questions

• RQ1 Do newer YOLO versions consistently outperform older ones across diverse domains?
• RQ2 What core innovations in each YOLO version translate into real-world gains or trade-offs?
• RQ3 How does YOLO performance vary across autonomous driving, agricultural, underwater, medical, and other domains?
• RQ4 What guidance can be offered for choosing YOLO versions in multi-domain, real-time applications?

Key findings

• New YOLO versions do not universally dominate older versions across all domains.
• The 33-dataset, multi-domain benchmark reveals domain-dependent performance gains and trade-offs.
• Innovations in newer versions yield varying improvements depending on domain characteristics and dataset specifics.
• ODverse33 provides a structured framework to compare YOLO versions beyond single-dataset benchmarks.
• The study offers practical insights for deploying real-time detectors in heterogeneous environments.