[论文解读] Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone

Hiroya Maeda, Yoshihide Sekimoto+3 more|arXiv (Cornell University)|Jan 29, 2018

Infrastructure Maintenance and Monitoring参考文献 26被引用 248

一句话总结

作者创建了一个通过智能手机捕捉的大规模道路损坏数据集，训练基于 SSD 的检测器来对八种损坏类型进行分类，并展示基于公开数据集与代码的智能手机实时推理。

ABSTRACT

Research on damage detection of road surfaces using image processing techniques has been actively conducted, achieving considerably high detection accuracies. Many studies only focus on the detection of the presence or absence of damage. However, in a real-world scenario, when the road managers from a governing body need to repair such damage, they need to clearly understand the type of damage in order to take effective action. In addition, in many of these previous studies, the researchers acquire their own data using different methods. Hence, there is no uniform road damage dataset available openly, leading to the absence of a benchmark for road damage detection. This study makes three contributions to address these issues. First, to the best of our knowledge, for the first time, a large-scale road damage dataset is prepared. This dataset is composed of 9,053 road damage images captured with a smartphone installed on a car, with 15,435 instances of road surface damage included in these road images. In order to generate this dataset, we cooperated with 7 municipalities in Japan and acquired road images for more than 40 hours. These images were captured in a wide variety of weather and illuminance conditions. In each image, we annotated the bounding box representing the location and type of damage. Next, we used a state-of-the-art object detection method using convolutional neural networks to train the damage detection model with our dataset, and compared the accuracy and runtime speed on both, using a GPU server and a smartphone. Finally, we demonstrate that the type of damage can be classified into eight types with high accuracy by applying the proposed object detection method. The road damage dataset, our experimental results, and the developed smartphone application used in this study are publicly available (https://github.com/sekilab/RoadDamageDetector/).

研究动机与目标

• 通过要求对可操作维护进行类型特定损坏分类来推动实际的道路损坏评估。
• 创建一个统一的、公开可用的道路损坏图像数据集，图像来自安装在汽车上的智能手机，在多样条件下捕获。
• 评估最先进的深度目标检测器在道路损坏检测与分类任务上的表现。
• 证明可以使用端到端深度学习在对移动友好的平台上以高准确率识别损坏类型。

提出的方法

• 开发一个包含9,053张标注的道路损坏图像（15,435个损坏实例）的 大规模数据集，使用安装在汽车上的智能手机捕获。
• 在数据集上训练和评估基于 SSD 的目标检测器（SSD Inception V2 和 SSD MobileNet）。
• 使用 600x600 图像，重新调整为 300x300 供 SSD 输入，学习率调度如所述（Inception V2：0.002，迭代10k次后衰减至 0.95；MobileNet：0.003，迭代10k次后衰减至 0.95）。
• 以 IOU 阈值 0.5 评估性能，并报告按类别的召回率、精度和准确率。
• 分析速度：基于GPU的推理与基于智能手机的推理（MobileNet更快；智能手机约 1500 ms）。
• 提供公开可获取的训练模型、代码，以及用于实时检测的智能手机应用。

实验结果

研究问题

• RQ1 端到端深度学习目标检测器是否能从智能手机捕捉的图像中准确分类八种道路损坏？
• RQ2 一个大规模、公开可获得的基于智能手机的道路损坏数据集是否可行且有用用于基准测试？
• RQ3 在该数据集上，基于 SSD 的检测器的检测性能（召回/精度）和推理速度如何，包括在设备上的智能手机推理？

主要发现

• 创建并发布了一个包含9,053张标注的道路损坏图像、15,435个损坏实例的数据集。
• SSD MobileNet 在各类别上都优于 SSD Inception V2，且具备显著的在设备上的推理能力。
• 在最佳配置下，某些损坏类型的召回率和精确度均超过75%。
• 基于智能手机的检测大约每张图像1.5秒，能够在移动车辆行驶中实现实时路边评估。
• 智能手机上的推理对检测类别的准确性与服务器端一致，使得在现场部署成为可能。