[论文解读] Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Mingxing Xu, Wenrui Dai+5 more|arXiv (Cornell University)|Jan 9, 2020

Traffic Prediction and Management Techniques参考文献 31被引用 265

一句话总结

本文提出 Spatial-Temporal Transformer Networks (STTNs)，对动态有向空间依赖和长期时序依赖进行建模，以提升长期交通流预测。STTNs 在 PeMS-BAY 和 PeMSD7(M) 上取得具有竞争力的、最先进的性能，尤其在较长预测区间。

ABSTRACT

Traffic forecasting has emerged as a core component of intelligent transportation systems. However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and dynamic spatial-temporal dependencies of traffic flows. In this paper, we propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) that leverages dynamical directed spatial dependencies and long-range temporal dependencies to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies with self-attention mechanism to capture realtime traffic conditions as well as the directionality of traffic flows. Furthermore, different spatial dependency patterns can be jointly modeled with multi-heads attention mechanism to consider diverse relationships related to different factors (e.g. similarity, connectivity and covariance). On the other hand, the temporal transformer is utilized to model long-range bidirectional temporal dependencies across multiple time steps. Finally, they are composed as a block to jointly model the spatial-temporal dependencies for accurate traffic prediction. Compared to existing works, the proposed model enables fast and scalable training over a long range spatial-temporal dependencies. Experiment results demonstrate that the proposed model achieves competitive results compared with the state-of-the-arts, especially forecasting long-term traffic flows on real-world PeMS-Bay and PeMSD7(M) datasets.

研究动机与目标

• 在高度动态的时空依赖下，解决准确的长期交通预测挑战。
• 引入一个空间变换器以捕捉时变的有向空间关系。
• 结合一个时序变换器以建模用于多步预测的长期时序依赖。
• 实现用于交通网络的联合时空建模的高效、可扩展训练。

提出的方法

• 定义包含空间变换器和时序变换器的时空块结构。
• 通过动态图卷积层和带门控融合机制的固定图卷积层来建模动态的空间依赖。
• 使用可学习的时空位置嵌入来结合拓扑和时间。
• 在空间和时序变换器中应用自注意力以捕捉长程依赖。
• 使用两层预测头进行并行的多步预测训练。
• 将 STTN 表述为具有消息传递语义的动态图神经网络。

实验结果

研究问题

• RQ1 如何为交通网络有效建模动态有向空间依赖？
• RQ2 时序变换器能否捕捉长期时序依赖以提升多步交通预测？
• RQ3 整合空间和时序变换器是否比固定空间布局模型在长期预测上具备更高的准确性？

主要发现

• STTN 在与最先进方法的比较中表现具有竞争力，在真实世界数据集 PeMSD7(M) 与 PEMS-BAY 的长期预测上取得显著提升。
• 空间变换器通过固定图卷积和动态图卷积来学习既受静态拓扑驱动又受动态交通条件驱动的空间模式。
• 时序变换器通过自注意力建模长期时序依赖，使得能够进行同时的多步预测。
• 门控机制有效融合来自固定和动态卷积的空间特征，提升鲁棒的特征表示。
• 该模型支持高效的并行训练和可扩展的长期依赖建模，长期预测方面超越了以往的方法。