[Paper Review] Comprehensive Performance Evaluation of YOLOv11, YOLOv10, YOLOv9, YOLOv8 and YOLOv5 on Object Detection of Power Equipment

Zijian He, Kai Wang+4 more|arXiv (Cornell University)|Nov 28, 2024

Industrial Vision Systems and Defect Detection4 citations

TL;DR

This study evaluates YOLOv5, YOLOv8, YOLOv9, YOLOv10, and YOLOv11 for power equipment object detection, demonstrating that YOLOv11 achieves the highest mAP (57.2%) and recall (64.8%) on a public dataset, outperforming prior versions through architectural enhancements like the C3K2 module and C2PSA attention mechanism.

ABSTRACT

With the rapid development of global industrial production, the demand for reliability in power equipment has been continuously increasing. Ensuring the stability of power system operations requires accurate methods to detect potential faults in power equipment, thereby guaranteeing the normal supply of electrical energy. In this article, the performance of YOLOv5, YOLOv8, YOLOv9, YOLOv10, and the state-of-the-art YOLOv11 methods was comprehensively evaluated for power equipment object detection. Experimental results demonstrate that the mean average precision (mAP) on a public dataset for power equipment was 54.4%, 55.5%, 43.8%, 48.0%, and 57.2%, respectively, with the YOLOv11 achieving the highest detection performance. Moreover, the YOLOv11 outperformed other methods in terms of recall rate and exhibited superior performance in reducing false detections. In conclusion, the findings indicate that the YOLOv11 model provides a reliable and effective solution for power equipment object detection, representing a promising approach to enhancing the operational reliability of power systems.

Motivation & Objective

• To evaluate the performance of recent YOLO variants (v5 to v11) in detecting power equipment for enhanced power system reliability.
• To identify the most accurate and robust object detection model for industrial power equipment under real-world conditions.
• To analyze the impact of architectural changes in YOLOv11 on detection accuracy, speed, and robustness in complex environments.
• To provide empirical evidence for adopting YOLOv11 as a state-of-the-art solution in power system monitoring and fault detection.

Proposed method

• The YOLOv11 model replaces the C2f module in YOLOv8 with a custom C3K2 module using two smaller convolutional layers instead of one large one, improving efficiency and feature extraction.
• The SPPF module from YOLOv8 is retained to enhance multi-scale feature learning and spatial pyramid pooling.
• A novel C2PSA module is introduced to integrate channel and spatial attention, enabling more effective feature representation through multi-head attention mechanisms.
• An adaptive anchor frame mechanism is implemented to automatically optimize anchor box configurations for different datasets, improving localization accuracy.
• The model is trained and validated on a public power equipment dataset using standard object detection metrics including mAP, precision, and recall.
• Experiments include comparative evaluation across five YOLO versions using mAP, precision, and recall on wires and transformers separately.

Experimental results

Research questions

• RQ1 How does YOLOv11 compare to YOLOv5, YOLOv8, YOLOv9, and YOLOv10 in terms of mAP and recall for power equipment object detection?
• RQ2 What architectural innovations in YOLOv11 contribute to its improved detection performance over previous versions?
• RQ3 To what extent does the adaptive anchor frame mechanism enhance detection accuracy across diverse power equipment types?
• RQ4 How do the C3K2 and C2PSA modules individually and collectively improve feature extraction and detection robustness?
• RQ5 Does YOLOv11 significantly reduce false positives and false negatives compared to earlier YOLO models in complex industrial settings?

Key findings

• YOLOv11 achieved the highest mean average precision (mAP) of 57.2% on the power equipment detection dataset, surpassing YOLOv5 (54.4%), YOLOv8 (55.5%), YOLOv9 (43.8%), and YOLOv10 (48.0%).
• YOLOv11 recorded the highest recall rate at 64.8%, indicating superior ability to detect true positive instances and minimize false negatives.
• On specific power equipment categories, YOLOv11 achieved 73.9% mAP for wires and 62.0% mAP for transformers, outperforming all other models.
• The C3K2 module in YOLOv11 reduced computational load through smaller kernel sizes while maintaining performance, enhancing inference speed.
• The integration of the C2PSA module significantly improved feature representation by jointly modeling channel and spatial dependencies.
• The adaptive anchor frame mechanism enabled automatic optimization of anchor boxes, contributing to higher localization accuracy across diverse datasets.