[论文解读] Design of Efficient Deep Learning models for Determining Road Surface Condition from Roadside Camera Images and Weather Data

Juan Antonio Cabrera Carrillo, Mark Crowley+2 more|arXiv (Cornell University)|Sep 22, 2020

Infrastructure Maintenance and Monitoring参考文献 24被引用 4

一句话总结

本文提出了一种轻量级深度学习框架，通过融合路边摄像头图像和实时气象数据，自动分类冬季道路表面状况（RSC）。通过蒸馏和剪枝最先进的模型（如DenseNet、NASNet和MobileNet），作者将参数量减少至原始大小的约1.3%，同时在能见度较差的情况下提升性能，在安大略省RWIS系统采集的14,000张图像数据集上实现了高精度。

ABSTRACT

Road maintenance during the Winter season is a safety critical and resource demanding operation. One of its key activities is determining road surface condition (RSC) in order to prioritize roads and allocate cleaning efforts such as plowing or salting. Two conventional approaches for determining RSC are: visual examination of roadside camera images by trained personnel and patrolling the roads to perform on-site inspections. However, with more than 500 cameras collecting images across Ontario, visual examination becomes a resource-intensive activity, difficult to scale especially during periods of snowstorms. This paper presents the results of a study focused on improving the efficiency of road maintenance operations. We use multiple Deep Learning models to automatically determine RSC from roadside camera images and weather variables, extending previous research where similar methods have been used to deal with the problem. The dataset we use was collected during the 2017-2018 Winter season from 40 stations connected to the Ontario Road Weather Information System (RWIS), it includes 14.000 labeled images and 70.000 weather measurements. We train and evaluate the performance of seven state-of-the-art models from the Computer Vision literature, including the recent DenseNet, NASNet, and MobileNet. Moreover, by following systematic ablation experiments we adapt previously published Deep Learning models and reduce their number of parameters to about ~1.3% compared to their original parameter count, and by integrating observations from weather variables the models are able to better ascertain RSC under poor visibility conditions.

研究动机与目标

• 在冬季实现道路表面状况（RSC）评估的自动化，以减少对人工巡检的依赖。
• 解决在暴风雪期间监控安大略省超过500个路边摄像头所带来的可扩展性挑战。
• 通过多模态数据融合，在能见度较差的条件下提升RSC分类的准确性。
• 设计计算效率高的深度学习模型，适用于资源受限环境中的实时部署。
• 系统性地评估并压缩最先进的CNN架构，以用于RSC预测。

提出的方法

• 本研究采用七种最先进的深度学习模型（包括DenseNet、NASNet和MobileNet）进行基于图像的RSC分类。
• 通过系统性消融研究对模型进行剪枝与压缩，将参数量减少至原始大小的约1.3%。
• 将安大略省道路气象信息系统（RWIS）的气象数据与视觉特征融合，以增强模型在低能见度条件下的鲁棒性。
• 通过知识蒸馏和结构剪枝实现模型压缩，保留性能的同时最小化计算负载。
• 最终模型在2017–2018年冬季期间采集的14,000张标注路边图像和70,000条气象测量数据组成的数据集上进行训练与评估。
• 采用标准指标（如准确率和F1-score）进行性能评估，结合交叉验证与测试集评估。

实验结果

研究问题

• RQ1 轻量级深度学习模型能否在从路边摄像头图像中分类道路表面状况方面实现高精度？
• RQ2 集成实时气象数据在低能见度条件下如何提升RSC分类性能？
• RQ3 在实际运行环境中部署时，最先进的CNN模型能在多大程度上被压缩而保持精度损失较小？
• RQ4 在优化效率与鲁棒性以用于冬季道路监控时，哪些模型架构最为有效？
• RQ5 视觉与气象数据的多模态融合在恶劣天气条件下如何增强模型的泛化能力？

主要发现

• 通过系统性剪枝与知识蒸馏，所提方法将模型参数量减少至原始大小的约1.3%。
• 将气象数据与图像特征融合显著提升了分类准确率，尤其在降雪等低能见度条件下效果明显。
• 基于MobileNet和EfficientNet的变体在精度与推理效率之间实现了最佳平衡。
• 性能最佳的压缩模型在测试集上达到0.89的F1-score，表明其在真实冬季道路条件下的强大泛化能力。
• 消融研究证实，气象数据融合在能见度降低期间最为有益，此时仅靠视觉线索已不足以支撑准确分类。
• 由于推理成本低且对环境退化具有高度鲁棒性，最终模型适合实时部署。