[Paper Review] A Computer Vision Enabled damage detection model with improved YOLOv5 based on Transformer Prediction Head

Arunabha M. Roy, Jayabrata Bhaduri|arXiv (Cornell University)|Mar 7, 2023

Infrastructure Maintenance and Monitoring22 citations

TL;DR

DenseSPH-YOLOv5 integrates DenseNet blocks, CBAM, extra tiny-object heads, and Swin Transformer Prediction Heads into YOLOv5 to achieve high-accuracy, real-time road damage detection on RDD-2018.

ABSTRACT

Objective:Computer vision-based up-to-date accurate damage classification and localization are of decisive importance for infrastructure monitoring, safety, and the serviceability of civil infrastructure. Current state-of-the-art deep learning (DL)-based damage detection models, however, often lack superior feature extraction capability in complex and noisy environments, limiting the development of accurate and reliable object distinction. Method: To this end, we present DenseSPH-YOLOv5, a real-time DL-based high-performance damage detection model where DenseNet blocks have been integrated with the backbone to improve in preserving and reusing critical feature information. Additionally, convolutional block attention modules (CBAM) have been implemented to improve attention performance mechanisms for strong and discriminating deep spatial feature extraction that results in superior detection under various challenging environments. Moreover, additional feature fusion layers and a Swin-Transformer Prediction Head (SPH) have been added leveraging advanced self-attention mechanism for more efficient detection of multiscale object sizes and simultaneously reducing the computational complexity. Results: Evaluating the model performance in large-scale Road Damage Dataset (RDD-2018), at a detection rate of 62.4 FPS, DenseSPH-YOLOv5 obtains a mean average precision (mAP) value of 85.25 %, F1-score of 81.18 %, and precision (P) value of 89.51 % outperforming current state-of-the-art models. Significance: The present research provides an effective and efficient damage localization model addressing the shortcoming of existing DL-based damage detection models by providing highly accurate localized bounding box prediction. Current work constitutes a step towards an accurate and robust automated damage detection system in real-time in-field applications.

Motivation & Objective

• Improve damage detection accuracy and localization for road damages in challenging environments.
• Preserve and reuse discriminative feature information to mitigate semantic loss in YOLOv5.
• Achieve real-time performance suitable for in-field inspection.
• Localize multi-scale damage with efficient feature fusion and attention mechanisms.

Proposed method

• Attach DenseNet blocks to CSPDarknet53 to preserve feature maps and enable feature reuse.
• Incorporate CBAM to refine channel and spatial feature maps for better attention.
• Add an extra tiny-object detection head to improve small-damage detection.
• Replace CNN heads with Swin Transformer Prediction Heads to leverage self-attention for multi-scale objects.
• Attach Spatial Pyramid Pooling (SPP) to the backbone for multi-scale receptive fields; use an improved PANet for multi-scale feature fusion.
• Use CIoU-based loss (with IoU, overlap, and aspect-ratio terms) and DIoU considerations for bounding box regression; apply NMS for final predictions.

Figure 1 : Sample images from RDD-2018 dataset ( Maeda et al., , 2018 ) : (a) to (g) correspond to each of the eight categories with the legends.

Experimental results

Research questions

• RQ1 Can DenseNet-augmented CSPDarknet53 improve feature preservation and detection accuracy over standard YOLOv5 on road damage data?
• RQ2 Does CBAM integration enhance detection performance in noisy, multi-object, and varied illumination conditions?
• RQ3 What is the impact of adding a dedicated tiny-object detection head and Swin Transformer Prediction Heads on multi-scale damage localization and speed?
• RQ4 How do SPP and an improved PANet affect contextual feature representation and localization accuracy across eight damage classes?
• RQ5 What are the model’s performance metrics (mAP, precision, F1, IoU, FPS) on the RDD-2018 dataset under real-world conditions?

Key findings

• DenseSPH-YOLOv5 achieves 85.25% mAP on RDD-2018 with 62.4 FPS.
• F1 score of 81.18% and precision of 89.51% were reported, indicating strong detection accuracy and reliable localization.
• DenseNet blocks and CSP enhancements improve feature preservation and reuse, boosting detection performance.
• CBAM improves attention over crowded/dense scenes, aiding discrimination between damage types.
• Swin Transformer Prediction Heads enhance multi-scale object detection while reducing computational cost.
• SPP and an improved PANet further enrich multi-scale feature representation and localization.

Figure 2 : Schematic of (a) YOLO object localization process for damage localization; (b) Schematic of CIoU offset regression for target BBs predictions.