[论文解读] Learning dynamic and hierarchical traffic spatiotemporal features with Transformer

Haoyang Yan, Xiaolei Ma|arXiv (Cornell University)|Apr 12, 2021

Traffic Prediction and Management Techniques被引用 24

一句话总结

本文提出 Traffic Transformer，通过多头注意力和遮掩注意力来建模长期预测的动态分层时空交通特征，超越现有最先进方法。

ABSTRACT

Traffic forecasting is an indispensable part of Intelligent transportation systems (ITS), and long-term network-wide accurate traffic speed forecasting is one of the most challenging tasks. Recently, deep learning methods have become popular in this domain. As traffic data are physically associated with road networks, most proposed models treat it as a spatiotemporal graph modeling problem and use Graph Convolution Network (GCN) based methods. These GCN-based models highly depend on a predefined and fixed adjacent matrix to reflect the spatial dependency. However, the predefined fixed adjacent matrix is limited in reflecting the actual dependence of traffic flow. This paper proposes a novel model, Traffic Transformer, for spatial-temporal graph modeling and long-term traffic forecasting to overcome these limitations. Transformer is the most popular framework in Natural Language Processing (NLP). And by adapting it to the spatiotemporal problem, Traffic Transformer hierarchically extracts spatiotemporal features through data dynamically by multi-head attention and masked multi-head attention mechanism, and fuse these features for traffic forecasting. Furthermore, analyzing the attention weight matrixes can find the influential part of road networks, allowing us to learn the traffic networks better. Experimental results on the public traffic network datasets and real-world traffic network datasets generated by ourselves demonstrate our proposed model achieves better performance than the state-of-the-art ones.

研究动机与目标

• 激发超越基于固定邻接的 GCN 交通预测的必要性。
• 提出一种基于 Transformer 的框架，以捕捉交通网络中的动态时空依赖关系。
• 通过数据驱动的注意力机制提取分层时空特征。
• 通过分析注意力权重实现对网络影响的解释。

提出的方法

• 将 Transformer 架构适配为用于交通数据的时空图建模。
• 使用多头注意力学习动态时空依赖，并使用遮蔽式多头注意力进行时间预测。
• 对来自注意力层的时空特征进行分层融合，以实现最终预测。
• 利用注意力权重矩阵识别对交通网络有影响的道路组件。
• 在公开数据集和真实世界数据集上展示出改进的预测性能。

实验结果

研究问题

• RQ1 Traffic Transformer 是否能够建模交通网络中的动态和分层时空依赖？
• RQ2 注意力权重矩阵如何反映对于预测有影响的道路网络部分？
• RQ3 所提出的模型相较于最先进方法是否提升长期交通预测？

主要发现

• Traffic Transformer 在公开交通数据集和真实世界数据集上获得的性能优于最先进方法。
• 注意力机制使从交通数据中分层提取时空特征成为可能。
• 对注意力权重的分析有助于识别对交通预测有影响的道路和网络区域。