[Paper Review] Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version

Razvan-Gabriel Cirstea, Bin Yang+3 more|arXiv (Cornell University)|Mar 29, 2022

Traffic Prediction and Management Techniques4 citations

TL;DR

This paper proposes a model-agnostic framework to enhance spatio-temporal awareness in traffic time series forecasting by generating location-specific and time-varying attention parameters via stochastic latent variables. By integrating these parameters into self-attention mechanisms and introducing a linear-complexity window attention, the method achieves state-of-the-art accuracy and efficiency across four real-world datasets, outperforming existing spatio-temporal agnostic models.

ABSTRACT

Traffic time series forecasting is challenging due to complex spatio-temporal dynamics time series from different locations often have distinct patterns; and for the same time series, patterns may vary across time, where, for example, there exist certain periods across a day showing stronger temporal correlations. Although recent forecasting models, in particular deep learning based models, show promising results, they suffer from being spatio-temporal agnostic. Such spatio-temporal agnostic models employ a shared parameter space irrespective of the time series locations and the time periods and they assume that the temporal patterns are similar across locations and do not evolve across time, which may not always hold, thus leading to sub-optimal results. In this work, we propose a framework that aims at turning spatio-temporal agnostic models to spatio-temporal aware models. To do so, we encode time series from different locations into stochastic variables, from which we generate location-specific and time-varying model parameters to better capture the spatio-temporal dynamics. We show how to integrate the framework with canonical attentions to enable spatio-temporal aware attentions. Next, to compensate for the additional overhead introduced by the spatio-temporal aware model parameter generation process, we propose a novel window attention scheme, which helps reduce the complexity from quadratic to linear, making spatio-temporal aware attentions also have competitive efficiency. We show strong empirical evidence on four traffic time series datasets, where the proposed spatio-temporal aware attentions outperform state-of-the-art methods in term of accuracy and efficiency. This is an extended version of "Towards Spatio-Temporal Aware Traffic Time Series Forecasting", to appear in ICDE 2022 [1], including additional experimental results.

Motivation & Objective

• To address the limitations of spatio-temporal agnostic deep learning models in traffic forecasting, which assume uniform parameters across locations and time periods.
• To enable dynamic, location-specific modeling of traffic patterns that vary across spatial locations and temporal windows.
• To develop an efficient attention mechanism that maintains high performance while reducing computational complexity.
• To empirically validate the framework's superiority in accuracy and efficiency over state-of-the-art methods on real-world traffic datasets.
• To demonstrate that learned stochastic variables effectively capture distinct spatial and temporal dynamics in traffic time series.

Proposed method

• The framework uses stochastic latent variables to generate location-specific and time-varying attention parameters, enabling dynamic adaptation to spatio-temporal patterns.
• It integrates these learned parameters into standard multi-head self-attention mechanisms, creating spatio-temporal aware attention layers.
• A novel window attention mechanism is proposed to reduce attention complexity from quadratic to linear by aggregating local windows of tokens.
• The aggregation function uses a learnable attention-based aggregator to combine proxy representations, outperforming simple mean aggregation.
• The model is trained end-to-end using a reconstruction loss on historical traffic time series, with parameters shared across time and space via stochastic parameter generation.
• t-SNE visualization is used to interpret the learned stochastic variables, confirming their ability to capture distinct spatial and temporal patterns.

Experimental results

Research questions

• RQ1 Can a model-agnostic framework effectively enhance spatio-temporal awareness in traffic forecasting by generating dynamic, location-specific attention parameters?
• RQ2 How does the proposed window attention mechanism maintain performance while reducing computational complexity compared to standard self-attention?
• RQ3 To what extent do the learned stochastic variables reflect actual spatial and temporal patterns in real traffic data?
• RQ4 Does the integration of spatio-temporal aware attention lead to measurable improvements in forecasting accuracy across diverse traffic datasets?
• RQ5 How does the choice of aggregation function impact the model’s performance and robustness?

Key findings

• The proposed spatio-temporal aware attention mechanism achieves state-of-the-art performance on four real-world traffic forecasting datasets, with MAE of 23.54, MAPE of 16.52, and RMSE of 36.87 on PEMS04.
• The window attention mechanism reduces computational complexity from quadratic to linear, enabling efficient processing of long historical sequences, as shown by stable training runtimes even at H=120.
• The model outperforms strong baselines such as STFGNN, AGCRN, and EnhanceNet in both accuracy and training efficiency, particularly on long-sequence forecasting tasks.
• t-SNE visualization confirms that learned stochastic variables capture distinct temporal dynamics, with different clusters corresponding to up/down trends in time series.
• The model's performance is significantly degraded when using a simple mean aggregator instead of the proposed learnable aggregator, validating the design choice.
• Sensors along the same road form spatially coherent clusters in the latent space, while sensors on opposite directions of the same road are separated, indicating effective spatial pattern capture.