[论文解读] PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction

Jiawei Jiang, Chengkai Han+2 more|arXiv (Cornell University)|Jan 19, 2023

Traffic Prediction and Management Techniques被引用 22

一句话总结

PDFormer 引入传播延迟感知的时空自注意力，配合短程与长程掩码以预测交通流量，达到最前沿的准确性和可解释的注意力图。

ABSTRACT

As a core technology of Intelligent Transportation System, traffic flow prediction has a wide range of applications. The fundamental challenge in traffic flow prediction is to effectively model the complex spatial-temporal dependencies in traffic data. Spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. However, GNN-based models have three major limitations for traffic prediction: i) Most methods model spatial dependencies in a static manner, which limits the ability to learn dynamic urban traffic patterns; ii) Most methods only consider short-range spatial information and are unable to capture long-range spatial dependencies; iii) These methods ignore the fact that the propagation of traffic conditions between locations has a time delay in traffic systems. To this end, we propose a novel Propagation Delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Specifically, we design a spatial self-attention module to capture the dynamic spatial dependencies. Then, two graph masking matrices are introduced to highlight spatial dependencies from short- and long-range views. Moreover, a traffic delay-aware feature transformation module is proposed to empower PDFormer with the capability of explicitly modeling the time delay of spatial information propagation. Extensive experimental results on six real-world public traffic datasets show that our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency. Moreover, we visualize the learned spatial-temporal attention map to make our model highly interpretable.

研究动机与目标

• 通过解决动态、长程和时延时空依赖性来推动准确的交通流预测。
• 用一个融合局部地理和全局语义邻域的时空自注意力模块来建模动态时空关系。
• 在预测过程中明确地纳入交通信息传播延迟。
• 捕捉动态时序模式并提供可解释的时空注意力机制。

提出的方法

• 提出基于时空自注意力的 PDFormer，包含三大核心组件：用于短程/长程时空依赖的 GeoSSA 与 SemSSA，以及用于时序动态的 TSA。
• 引入一个带延迟感知的特征变换（DFT），通过使用 k-Shape 聚类和 DTW 基于语义邻居来用历史模式记忆增强键表示。
• 在注意力计算中使用两个图掩码矩阵 M_geo 与 M_sem，以强调局部地理关系和全局语义关系。
• 采用多头异质注意力融合，在前馈网络和残差连接之前把 GeoSSA、SemSSA 和 TSA 的输出进行融合。
• 数据嵌入层包含图拉普拉斯嵌入和时间周期嵌入，以注入结构和周期信息。
• 输出层使用跳跃连接和1x1卷积实现多步预测，无需递归解码。

实验结果

研究问题

• RQ1 如何为交通数据有效建模动态时空依赖？

主要发现

• PDFormer 在六个真实交通数据集（三个基于图的和三个基于网格的）上在MAE、MAPE和RMSE上实现了最先进的性能。
• 在消融实验中，SSA 优于 GCN，因为它能够捕捉动态和长程的时空依赖。
• 两种掩码矩阵（M_geo 和 M_sem）通过引导注意力关注相关的局部和远距离但相似的区域显著提升性能。
• 带延迟感知的特征变换通过显式建模空间传播延迟来改进预测。
• PDFormer 提供具有竞争力的训练和推理效率，在若干效率指标上优于基线。
• 时空注意力的可视化展示了可解释性，显示关注于邻近且语义相似的区域。