[论文解读] Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values

Zhiyong Cui, Ruimin Ke+2 more|arXiv (Cornell University)|May 24, 2020

Traffic Prediction and Management Techniques参考文献 53被引用 37

一句话总结

本文提出 SBU-LSTM，一种堆叠架构，结合 双向 LSTM 与 插补 (BDLSTM-I) 和 单向 LSTM 层，通过整合的 插补 单元来预测整个网络的交通速度并处理缺失数据。

ABSTRACT

Short-term traffic forecasting based on deep learning methods, especially recurrent neural networks (RNN), has received much attention in recent years. However, the potential of RNN-based models in traffic forecasting has not yet been fully exploited in terms of the predictive power of spatial-temporal data and the capability of handling missing data. In this paper, we focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models. A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assist the design of neural network structures for traffic state forecasting. As a key component of the architecture, the bidirectional LSTM (BDLSM) is exploited to capture the forward and backward temporal dependencies in spatiotemporal data. To deal with missing values in spatial-temporal data, we also propose a data imputation mechanism in the LSTM structure (LSTM-I) by designing an imputation unit to infer missing values and assist traffic prediction. The bidirectional version of LSTM-I is incorporated in the SBU-LSTM architecture. Two real-world network-wide traffic state datasets are used to conduct experiments and published to facilitate further traffic prediction research. The prediction performance of multiple types of multi-layer LSTM or BDLSTM models is evaluated. Experimental results indicate that the proposed SBU-LSTM architecture, especially the two-layer BDLSTM network, can achieve superior performance for the network-wide traffic prediction in both accuracy and robustness. Further, comprehensive comparison results show that the proposed data imputation mechanism in the RNN-based models can achieve outstanding prediction performance when the model's input data contains different patterns of missing values.

研究动机与目标

• 推动改进的短期网络范围交通预测，能够处理传感器数据缺失。
• 提出带有插补单元的 LSTM 变体（LSTM-I），在预测模型中推断缺失值。
• 引入一个堆叠架构（SBU-LSTM），将双向和单向 LSTM 组件结合起来，以获得更好的时空特征学习。
• 在真实世界数据集上评估模型性能，并分析模型容量与复杂性之间的权衡。
• 发布并分享 LOOP-SEA 数据集，以促进进一步研究。

提出的方法

• 将 X 定义为一个 T x D 的交通状态序列，并附带一个掩码矩阵 M，指示缺失值。
• 引入带有插补单元的 LSTM-I，通过 C_{t-1} 和 h_{t-1} 推断缺失的 x_t，并通过掩码机制更新输入。
• 在 LSTM-I 损失中加入一个正则化项，惩罚插补误差（lambda * sum of |x_t - xhat_t|）。
• 扩展为带插补的双向 LSTM（BDLSTM-I），通过前向和后向传递来插补缺失值，并通过平均运算符进行组合。
• 将 BDLSTM-I 与 LSTM/LBDSTM 层堆叠成灵活的 SBU-LSTM 架构，其中第一层在存在缺失数据时为 BDLSTM-I，随后根据需要添加其他层。
• 使用带 Adam 的 MSE 训练、早停和学习率衰减来训练；在不同缺失率下，针对随机和非随机缺失数据模式进行评估。

实验结果

研究问题

• RQ1 结合双向时序依赖的堆叠架构是否能提升相较于单向或单层模型的网络范围交通预测准确性？
• RQ2 在 RNN（LSTM-I/BDLSTM-I）中集成数据插补单元是否能在传感器数据缺失时提升鲁棒性和预测准确性？
• RQ3 模型深度和层类型（BDLSTM-I 与 LSTM/LBDSTM）的影响对预测性能和计算权衡是什么？
• RQ4 在具有不同缺失数据模式的真实世界网络级交通数据集上，该方法的表现如何？

主要发现

• 基于 BDLSTM 的架构，尤其是两层的 BDLSTM，在 LOOP-SEA 和 PEMS-BAY 数据集上实现了最佳预测精度。
• 两层 BDLSTM 通常优于单层模型和更深的架构，指出了模型容量与复杂度之间的一个平衡点。
• BDLSTM-I（具备缺失数据处理）在输入数据包含缺失值时通常表现更好，插补集成在训练目标中。
• 所提议的数据插补机制（LSTM-I/BDLSTM-I）在各种缺失值模式（随机和非随机）下提高了预测鲁棒性。
• 结果表明，以 BDLSTM 为基础的堆叠在作为网络级交通预测的最终层时更有效，并且该架构可以灵活地扩展为带有 LSTM/BDLSTM 层的结构。