[论文解读] Advances in deep learning methods for pavement surface crack detection and identification with visible light visual images

Kai-liang Lu|arXiv (Cornell University)|Dec 29, 2020

Infrastructure Maintenance and Monitoring参考文献 16被引用 8

一句话总结

本文综述了使用可见光图像进行路面裂缝检测的深度学习方法，重点聚焦于在嵌入式平台上实现实时、高精度的像素级分割。结果表明，轻量级模型（如基于MobileNet的迁移学习）在单张图像上的推理速度可低于10ms，对简单样本的准确率超过99.8%，对挑战性样本的准确率可达97.5%（FPCNet在SDNET2018上的表现），标志着可部署的高性能裂缝检测系统取得了重大进展。

ABSTRACT

Compared to NDT and health monitoring method for cracks in engineering structures, surface crack detection or identification based on visible light images is non-contact, with the advantages of fast speed, low cost and high precision. Firstly, typical pavement (concrete also) crack public data sets were collected, and the characteristics of sample images as well as the random variable factors, including environmental, noise and interference etc., were summarized. Subsequently, the advantages and disadvantages of three main crack identification methods (i.e., hand-crafted feature engineering, machine learning, deep learning) were compared. Finally, from the aspects of model architecture, testing performance and predicting effectiveness, the development and progress of typical deep learning models, including self-built CNN, transfer learning(TL) and encoder-decoder(ED), which can be easily deployed on embedded platform, were reviewed. The benchmark test shows that: 1) It has been able to realize real-time pixel-level crack identification on embedded platform: the entire crack detection average time cost of an image sample is less than 100ms, either using the ED method (i.e., FPCNet) or the TL method based on InceptionV3. It can be reduced to less than 10ms with TL method based on MobileNet (a lightweight backbone base network). 2) In terms of accuracy, it can reach over 99.8% on CCIC which is easily identified by human eyes. On SDNET2018, some samples of which are difficult to be identified, FPCNet can reach 97.5%, while TL method is close to 96.1%. To the best of our knowledge, this paper for the first time comprehensively summarizes the pavement crack public data sets, and the performance and effectiveness of surface crack detection and identification deep learning methods for embedded platform, are reviewed and evaluated.

研究动机与目标

• 系统总结公开可用的路面裂缝数据集及其特性，包括环境因素和噪声相关的变异性。
• 比较手工特征工程、机器学习与深度学习在裂缝检测中的优缺点。
• 评估深度学习模型（特别是自建CNN、迁移学习（TL）和编码器-解码器（ED）架构）在嵌入式平台上的性能、效率与可部署性。
• 在实际现场条件下，建立基于可见光成像的实时、高精度裂缝检测基准。

提出的方法

• 整理并分析主要公开的路面裂缝数据集（包括CCIC和SDNET2018），以表征图像变异性和噪声因素。
• 评估三种主要方法：手工特征、传统机器学习与深度学习，重点关注其在准确率与复杂度之间的权衡。
• 实现并基准测试三种深度学习架构：自建CNN、基于InceptionV3和MobileNet主干的迁移学习，以及FPCNet等编码器-解码器模型。
• 通过轻量化网络设计优化模型推理速度与准确率，尤其针对嵌入式部署优化基于MobileNet的迁移学习。
• 使用标准指标在基准数据集上评估性能，重点关注推理时间与像素级分割准确率。
• 在嵌入式平台上进行实时测试，以验证其在实际应用中的部署可行性。

实验结果

研究问题

• RQ1 公开可用的路面裂缝图像数据集的关键特性与挑战是什么，包括环境与噪声变化？
• RQ2 在裂缝检测中，手工特征工程、机器学习与深度学习方法在准确率与计算效率方面的表现如何比较？
• RQ3 深度学习模型能否在适用于现场部署的低功耗嵌入式平台上实现实时、高精度的裂缝检测？
• RQ4 在CCIC和SDNET2018等基准数据集上，最先进深度学习模型（特别是FPCNet和基于MobileNet的迁移学习）的性能如何？
• RQ5 在嵌入式部署场景下，推理速度与检测准确率之间的权衡关系是什么？

主要发现

• FPCNet编码器-解码器模型在具有挑战性的SDNET2018数据集上实现了97.5%的像素级裂缝检测准确率。
• 基于MobileNet的迁移学习模型实现了每张图像低于10ms的推理时间，可在嵌入式平台上实现实时处理。
• 基于InceptionV3的迁移学习模型实现了每张图像低于100ms的推理时间，适用于实时应用。
• 在CCIC数据集中，裂缝易于识别，最佳模型的准确率超过99.8%。
• 本研究首次建立了基于深度学习的路面裂缝检测在嵌入式系统上的全面基准，展示了高准确率与低延迟。
• 结果证实，轻量级深度学习模型（尤其是基于MobileNet主干的模型）在真实世界基础设施检测系统中既具备高度准确性，又具备良好的可部署性。