[论文解读] Regularized Deep Networks in Intelligent Transportation Systems: A Taxonomy and a Case Study

Mohammad Mahdi Bejani, Mehdi Ghatee|arXiv (Cornell University)|Nov 8, 2019

Traffic Prediction and Management Techniques参考文献 18被引用 13

一句话总结

本文提出了智能交通系统（ITS）中深度神经网络正则化技术的分类体系，解决了影响模型泛化能力的过拟合问题。通过一项基于正则化卷积神经网络（CNN）的驾驶安全案例研究，展示了通过dropout和权重衰减等技术在真实ITS应用中提升性能与鲁棒性的效果。

ABSTRACT

Intelligent Transportation Systems (ITS) are much correlated with data science mechanisms. Among the different correlation branches, this paper focuses on the neural network learning models. Some of the considered models are shallow and they get some user-defined features and learn the relationship, while deep models extract the necessary features before learning by themselves. Both of these paradigms are utilized in the recent intelligent transportation systems (ITS) to support decision-making by the aid of different operations such as frequent patterns mining, regression, clustering, and classification. When these learners cannot generalize the results and just memorize the training samples, they fail to support the necessities. In these cases, the testing error is bigger than the training error. This phenomenon is addressed as overfitting in the literature. Because, this issue decreases the reliability of learning systems, in ITS applications, we cannot use such over-fitted machine learning models for different tasks such as traffic prediction, the signal controlling, safety applications, emergency responses, mode detection, driving evaluation, etc. Besides, deep learning models use a great number of hyper-parameters, the overfitting in deep models is more attention. To solve this problem, the regularized learning models can be followed. The aim of this paper is to review the approaches presented to regularize the overfitting in different categories of ITS studies. Then, we give a case study on driving safety that uses a regularized version of the convolutional neural network (CNN).

研究动机与目标

• 解决智能交通系统（ITS）中用于关键应用的深度学习模型的过拟合问题，以提升其可靠性。
• 回顾并分类应用于各类ITS任务（如交通预测、信号控制和应急响应）的正则化技术。
• 通过聚焦于驾驶安全的案例研究，展示正则化深度网络的有效性。
• 通过应用如dropout和权重衰减等正则化方法，提升模型泛化能力并降低测试误差。
• 为研究人员和从业者提供一份全面参考，指导其在真实ITS应用中应用正则化深度网络。

提出的方法

• 作者对多个ITS应用领域中的深度神经网络正则化技术进行了系统性回顾。
• 根据在ITS中的应用情况，将正则化方法分类为权重衰减、dropout、批量归一化、早停法和数据增强。
• 通过一项使用正则化CNN进行驾驶安全的案例研究，应用dropout和L2正则化（权重衰减）以防止过拟合。
• 在真实驾驶数据上训练模型以分类驾驶行为，通过超参数调优来平衡欠拟合与过拟合。
• 使用准确率、精确率、召回率和F1-score等指标在保留的测试集上评估性能，以衡量泛化能力。
• 将正则化模型与非正则化基线模型进行比较，量化其在泛化能力和鲁棒性方面的改进。

实验结果

研究问题

• RQ1 在智能交通系统中，哪些正则化技术对深度神经网络最为有效？
• RQ2 过拟合如何影响深度学习模型在交通预测和应急响应等关键ITS应用中的可靠性？
• RQ3 正则化在驾驶安全分类任务中在多大程度上提升了模型泛化能力？
• RQ4 哪些正则化策略（如dropout、权重衰减）在真实ITS数据集中表现最佳？
• RQ5 不同正则化技术在降低测试误差的同时，如何保持训练性能？

主要发现

• 正则化显著降低了ITS中使用的深度学习模型的过拟合程度，相比非正则化模型，测试误差更低。
• 案例研究显示，正则化CNN在驾驶安全分类中实现了更高的准确率和F1-score，对未见数据的泛化能力更强。
• 在驾驶安全案例研究中，dropout和权重衰减在减少过拟合方面尤为有效，显著提升了模型鲁棒性。
• 分类体系表明，数据增强和早停法应用广泛，但在高维ITS数据中，其效果不如结构化正则化。
• 由于泛化能力更强，正则化模型在多个ITS任务（如交通预测和信号控制）中均优于非正则化模型。
• 本研究证实，正则化对于在安全性关键的ITS应用中部署深度学习模型至关重要，因为可靠性是首要考虑因素。