[论文解读] A Computer Vision Enabled damage detection model with improved YOLOv5 based on Transformer Prediction Head

Arunabha M. Roy, Jayabrata Bhaduri|arXiv (Cornell University)|Mar 7, 2023

Infrastructure Maintenance and Monitoring被引用 22

一句话总结

DenseSPH-YOLOv5 将 DenseNet 块、CBAM、额外的极小对象检测头和 Swin Transformer 预测头集成到 YOLOv5 中，以在 RDD-2018 上实现高精度、实时路面损坏检测。

ABSTRACT

Objective:Computer vision-based up-to-date accurate damage classification and localization are of decisive importance for infrastructure monitoring, safety, and the serviceability of civil infrastructure. Current state-of-the-art deep learning (DL)-based damage detection models, however, often lack superior feature extraction capability in complex and noisy environments, limiting the development of accurate and reliable object distinction. Method: To this end, we present DenseSPH-YOLOv5, a real-time DL-based high-performance damage detection model where DenseNet blocks have been integrated with the backbone to improve in preserving and reusing critical feature information. Additionally, convolutional block attention modules (CBAM) have been implemented to improve attention performance mechanisms for strong and discriminating deep spatial feature extraction that results in superior detection under various challenging environments. Moreover, additional feature fusion layers and a Swin-Transformer Prediction Head (SPH) have been added leveraging advanced self-attention mechanism for more efficient detection of multiscale object sizes and simultaneously reducing the computational complexity. Results: Evaluating the model performance in large-scale Road Damage Dataset (RDD-2018), at a detection rate of 62.4 FPS, DenseSPH-YOLOv5 obtains a mean average precision (mAP) value of 85.25 %, F1-score of 81.18 %, and precision (P) value of 89.51 % outperforming current state-of-the-art models. Significance: The present research provides an effective and efficient damage localization model addressing the shortcoming of existing DL-based damage detection models by providing highly accurate localized bounding box prediction. Current work constitutes a step towards an accurate and robust automated damage detection system in real-time in-field applications.

研究动机与目标

• 在具有挑战性的环境中提高路面损坏检测的准确性和定位能力。
• 保留并重用具有辨识性的特征信息，以缓解 YOLOv5 的语义损失。
• 实现适用于现场检查的实时性能。
• 通过高效的特征融合和注意力机制对多尺度损坏进行定位。

提出的方法

• 将 DenseNet 块附加到 CSPDarknet53 以保留特征图并实现特征重用。
• 引入 CBAM 来细化通道和空间特征图，以获得更好的注意力。
• 新增一个额外的极小对象检测头以提升对小尺寸损伤的检测。
• 用 Swin Transformer 预测头替代 CNN 头以利用自注意力处理多尺度对象。
• 在骨干网络上增加空间金字塔池化（SPP），以获得多尺度感受野；使用改进的 PANet 进行多尺度特征融合。
• 在边界框回归中使用基于 CIoU 的损失（包含 IoU、重叠度和纵横比项）；结合 DIoU 考虑；对最终预测应用 NMS。

Figure 1 : Sample images from RDD-2018 dataset ( Maeda et al., , 2018 ) : (a) to (g) correspond to each of the eight categories with the legends.

实验结果

研究问题

• RQ1 在路面损坏数据上，带 DenseNet 增强的 CSPDarknet53 是否能比标准 YOLOv5 提高特征保留和检测准确性？
• RQ2 在嘈杂、多对象和光照条件各异的情况下，CBAM 的整合是否提升检测性能？
• RQ3 添加专门的极小对象检测头和 Swin Transformer 预测头对多尺度损伤定位和速度的影响是什么？
• RQ4 SPP 和改进的 PANet 如何影响八类损伤在上下文特征表征和定位精度上的表现？
• RQ5 在真实世界条件下，该模型在 RDD-2018 数据集上的性能指标（mAP、精确度、F1、IoU、FPS）是多少？

主要发现

• DenseSPH-YOLOv5 在 RDD-2018 上达到 85.25% mAP，FPS 为 62.4。
• 报告的 F1 分数为 81.18%，精确度为 89.51%，表明检测精度高且定位可靠。
• DenseNet 块和 CSP 的强化提高了特征保留与重用，从而提升检测性能。
• CBAM 在拥挤/密集场景中提升了注意力，有助于区分损坏类型。
• Swin Transformer 预测头在提升多尺度对象检测的同时降低了计算成本。
• SPP 和改进的 PANet 进一步丰富了多尺度特征表示和定位。

Figure 2 : Schematic of (a) YOLO object localization process for damage localization; (b) Schematic of CIoU offset regression for target BBs predictions.