[论文解读] A Novel Variable Selection Method based on Frequent Pattern Tree for Real-time Traffic Accident Risk Prediction

Lei Lin, Qian Wang+1 more|arXiv (Cornell University)|Jan 20, 2017

Traffic Prediction and Management Techniques被引用 7

一句话总结

本文提出了一种基于频繁模式（FP）树算法与新型相对对象纯度比（ROPR）准则的新型变量选择方法，用于识别实时交通事故风险的关键预测变量。在2005年I-64州际公路（弗吉尼亚州）数据集上评估，基于FP树的变量选择方法在k-NN和贝叶斯网络模型中均优于基于随机森林的变量选择方法，预测准确率达到61.11%，误报率为38.16%。

ABSTRACT

Traffic accident data are usually noisy, contain missing values, and heterogeneous. How to select the most important variables to improve real-time traffic accident risk prediction has become a concern of many recent studies. This paper proposes a novel variable selection method based on the Frequent Pattern tree (FP tree) algorithm. First, all the frequent patterns in the traffic accident dataset are discovered. Then for each frequent pattern, a new criterion, called the Relative Object Purity Ratio (ROPR) which we proposed, is calculated. This ROPR is added to the importance score of the variables that differentiates one frequent pattern from the others. To test the proposed method, a dataset was compiled from the traffic accidents records detected by only one detector on interstate highway I-64 in Virginia in 2005. This data set was then linked to other variables such as real-time traffic information and weather conditions. Both the proposed method based on the FP tree algorithm, as well as the widely utilized, random forest method, were then used to identify the important variables or the Virginia data set. The results indicate that there are some differences between the variables deemed important by the FP tree and those selected by the random forest method. Following this, two baseline models (i.e. a nearest neighbor (k-NN) method and a Bayesian network) were developed to predict accident risk based on the variables identified by both the FP tree method and the random forest method. The results show that the models based on the variable selection using the FP tree performed better than those based on the random forest method for several versions of the k-NN and Bayesian network models.The best results were derived from a Bayesian network model using variables from FP tree. That model could predict 61.11% of accidents accurately, while having a false alarm rate of 38.16%.

研究动机与目标

• 解决实时风险预测中交通事故数据存在噪声、不完整和异构性的问题。
• 开发一种新型变量选择方法，从复杂交通数据集中识别出最相关的预测变量。
• 通过数据驱动的、基于模式的方法，提升实时交通事故风险预测模型的准确性和可靠性。
• 将所提方法与随机森林等成熟技术在预测建模中的表现进行对比。

提出的方法

• 使用FP树算法从交通事故数据集中挖掘频繁模式，捕捉事故事件中共同出现的变量。
• 为每个频繁模式计算一种新型相对对象纯度比（ROPR），以量化其组成变量的区分能力。
• 将ROPR值整合进变量重要性评分，优先选择能最好区分高风险与低风险模式的变量。
• 利用所选变量训练基线模型：k-最近邻（k-NN）和贝叶斯网络，用于风险预测。
• 将基于FP树选择变量训练的模型性能，与基于随机森林选择变量的模型进行对比。
• 在2005年I-64州际公路（弗吉尼亚州）的真实世界数据集上评估该方法，数据集还整合了实时交通与天气信息。

实验结果

研究问题

• RQ1 所提出的基于FP树的变量选择方法结合ROPR，在识别交通事故风险关键预测变量方面的有效性如何？
• RQ2 基于FP树选择变量的预测模型性能，与基于随机森林选择变量的模型相比如何？
• RQ3 采用所提方法选择变量的模型，其预测准确率和误报率分别是多少？
• RQ4 与传统变量选择技术相比，基于FP树的方法是否能提升实时交通事故风险的检测能力？

主要发现

• 基于FP树方法选择变量训练的贝叶斯网络模型，在交通事故发生预测中达到最高的准确率61.11%。
• 同一贝叶斯网络模型的误报率为38.16%，表明其在检测率与误报之间实现了良好平衡。
• 在k-NN和贝叶斯网络模型的多种配置下，基于FP树选择变量的模型性能均优于基于随机森林选择变量的模型。
• FP树方法识别出的显著变量集合与随机森林存在差异，表明其能提供关于风险因素的互补性洞见。
• ROPR准则通过强调能区分事故与非事故事件的模式，有效提升了变量重要性评估的准确性。
• 所提方法在复杂、含噪的交通数据集中，展现出优于传统方法的实时事故风险预测性能。