[论文解读] Short-term passenger flow prediction for multi-traffic modes: A Transformer and residual network based multi-task learning method

Jian Yang, Jinlei Zhang+3 more|arXiv (Cornell University)|Feb 27, 2022

Traffic Prediction and Management Techniques被引用 4

一句话总结

本文提出Res-Transformer，一种结合改进的Transformer层与残差网络的多任务学习模型，用于预测地铁、出租车和公交车等多种交通模式的短时乘客流量。通过联合建模时空依赖性和跨模式相关性，该模型在真实北京数据集上的表现优于基线模型，展现出更高的准确性和鲁棒性。

ABSTRACT

With the prevailing of mobility as a service (MaaS), it becomes increasingly important to manage multi-traffic modes simultaneously and cooperatively. As an important component of MaaS, short-term passenger flow prediction for multi-traffic modes has thus been brought into focus. It is a challenging problem because the spatiotemporal features of multi-traffic modes are critically complex. Moreover, the passenger flows of multi-traffic modes differentiate and fluctuate significantly. To solve these problems, this paper proposes a multitask learning-based model, called Res-Transformer, for short-term inflow prediction of multi-traffic modes (subway, taxi, and bus). Each traffic mode is treated as a single task in the model. The Res-Transformer consists of two parts: (1) several modified Transformer layers comprising the conv-Transformer layer and the multi-head attention mechanism, which helps to extract the spatial and temporal features of multi-traffic modes, (2) the structure of residual network is utilized to obtain the correlations of different traffic modes and prevent gradient vanishing, gradient explosion, and overfitting. The Res-Transformer model is evaluated on two large-scale real-world datasets from Beijing, China. One is the region of a traffic hub and the other is the region of a residential area. Experiments are conducted to compare the performance of the proposed model with several baseline models. Results prove the effectiveness and robustness of the proposed method. This paper can give critical insights into the short-term inflow prediction for multi-traffic modes.

研究动机与目标

• 为在统一框架中解决跨多种交通模式（地铁、出租车、公交车）的短时乘客流量预测挑战。
• 建模多交通系统中复杂的时空模式与模式间相关性。
• 通过缓解深度学习架构中的梯度消失与过拟合问题，提升预测准确性和模型稳定性。
• 在来自不同城市区域（交通枢纽与居民区）的真实世界数据集上评估模型性能。

提出的方法

• 模型采用改进的Transformer层，结合卷积-Transformer与多头注意力机制，从多交通流量数据中提取时空特征。
• 集成残差网络结构以增强特征学习能力，稳定训练过程，并减少多任务下的过拟合。
• 在多任务学习框架中，将每种交通模式（地铁、出租车、公交车）视为独立但联合优化的任务。
• 通过共享底层特征实现跨任务知识迁移，同时为每个任务保留特定的预测头以实现个体化预测。
• 模型在来自北京的两个大规模真实世界数据集上进行端到端训练，覆盖不同城市区域。
• 残差连接的使用有助于防止深层网络优化过程中的梯度消失与梯度爆炸问题。

实验结果

研究问题

• RQ1 统一的深度学习模型能否有效同时预测多种交通模式的短时乘客流量？
• RQ2 时空依赖性与模式间相关性在多交通系统中对预测性能有何影响？
• RQ3 通过共享与特定任务组件相结合的多任务学习，在多大程度上提升了预测准确性和鲁棒性？
• RQ4 残差网络的集成在多交通流量预测中如何增强训练稳定性和泛化能力？

主要发现

• Res-Transformer模型在所有交通模式下，均优于多个基线模型，其均方根误差（RMSE）和平均绝对误差（MAE）均更低。
• 该模型在两个不同城市区域（交通枢纽与居民区）均表现出稳健性能，表明其具备强大的泛化能力。
• 与单任务基线相比，多任务学习设置显著提升了预测准确性，尤其在低流量模式（如出租车）中表现更优。
• 消融实验确认，改进的Transformer与残差网络组件均对性能提升有显著贡献。
• 该模型能有效捕捉不同交通网络之间的长程时间依赖与空间相关性。
• 残差连接的集成降低了训练不稳定性，有效防止了过拟合，尤其在更深的网络结构中表现突出。