[Paper Review] Road Damages Detection and Classification with YOLOv7

Vung Pham, Van-Du Nguyen+1 more|arXiv (Cornell University)|Oct 31, 2022

Infrastructure Maintenance and Monitoring4 citations

TL;DR

This paper proposes an efficient road damage detection and classification system using YOLOv7 with coordinate attention and label smoothing, trained on road images collected from Google Street View. It achieved an F1 score of 81.7% on U.S. test data and ranked 3rd in the CRDDC2022 challenge, demonstrating high accuracy and scalability for automated infrastructure monitoring.

ABSTRACT

Maintaining the roadway infrastructure is one of the essential factors in enabling a safe, economic, and sustainable transportation system. Manual roadway damage data collection is laborious and unsafe for humans to perform. This area is poised to benefit from the rapid advance and diffusion of artificial intelligence technologies. Specifically, deep learning advancements enable the detection of road damages automatically from the collected road images. This work proposes to collect and label road damage data using Google Street View and use YOLOv7 (You Only Look Once version 7) together with coordinate attention and related accuracy fine-tuning techniques such as label smoothing and ensemble method to train deep learning models for automatic road damage detection and classification. The proposed approaches are applied to the Crowdsensing-based Road Damage Detection Challenge (CRDDC2022), IEEE BigData 2022. The results show that the data collection from Google Street View is efficient, and the proposed deep learning approach results in F1 scores of 81.7% on the road damage data collected from the United States using Google Street View and 74.1% on all test images of this dataset.

Motivation & Objective

• To develop an efficient, automated pipeline for collecting and labeling road damage data using Google Street View.
• To improve object detection accuracy for road damage classification using state-of-the-art YOLOv7 with attention mechanisms and hyperparameter tuning.
• To apply and evaluate the proposed model on the CRDDC2022 benchmark dataset for real-world performance assessment.
• To enhance detection robustness through data augmentation, label smoothing, and ensemble learning techniques.
• To contribute a publicly available dataset and model code to support future research in automated road condition monitoring.

Proposed method

• Collected road images from Google Street View using a systematic data collection pipeline for global coverage and frequent updates.
• Applied YOLOv7 as the base object detection model, leveraging its speed and accuracy for real-time inference.
• Integrated coordinate attention modules to enhance spatial feature representation and improve localization precision.
• Employed label smoothing and data augmentation techniques to reduce overfitting and improve generalization.
• Used ensemble learning by combining multiple YOLOv7 models to boost prediction reliability and F1 score.
• Fine-tuned hyperparameters using mAP@0.5 on validation sets and evaluated final models using F1 score on test sets.

Experimental results

Research questions

• RQ1 Can Google Street View be effectively leveraged to collect and label large-scale, diverse road damage images for training deep learning models?
• RQ2 How does the integration of coordinate attention into YOLOv7 improve detection accuracy for road damage classification?
• RQ3 To what extent do label smoothing and ensemble methods enhance the robustness and F1 score of YOLOv7 in road damage detection?
• RQ4 How does the proposed model perform across different geographic regions with varying road damage types and image quality?
• RQ5 Can the proposed approach achieve state-of-the-art performance in real-world benchmarks like CRDDC2022?

Key findings

• The proposed data collection method using Google Street View proved efficient and scalable, enabling access to a large volume of up-to-date road images from diverse global locations.
• The YOLOv7 model with coordinate attention achieved an F1 score of 81.7% on the United States test set, demonstrating strong performance on longitudinal and common damage types.
• The model achieved a test F1 score of 74.1% on the overall dataset, ranking 3rd out of 51 teams in the CRDDC2022 challenge.
• The highest performance was observed in the U.S. and Japan test sets, correlating with higher availability of usable training images and more distinguishable damage types.
• The use of label smoothing and ensemble techniques contributed to improved generalization and robustness, especially in low-data regions like India and Norway.
• Despite computational constraints, the approach demonstrated that even standard YOLOv7 configurations can achieve high accuracy, with potential for further gains using heavier variants like YOLOv7-X or YOLOv7-E6.