[Paper Review] STGRAT: A Spatio-Temporal Graph Attention Network for Traffic Forecasting.

Cheonbok Park, Chunggi Lee+6 more|arXiv (Cornell University)|Nov 29, 2019

Traffic Prediction and Management TechniquesEngineering38 citations

TL;DR

This paper proposes STGRAT, a spatio-temporal graph attention network that models dynamic spatial dependencies and temporal patterns in traffic networks using spatial attention, temporal attention, and spatial sentinel vectors. It achieves state-of-the-art performance, particularly in high-variability conditions like rush hours, by adaptively focusing on relevant road segments and temporal dynamics.

ABSTRACT

Predicting the road traffic speed is a challenging task due to different types of roads, abrupt speed changes, and spatial dependencies between roads, which requires the modeling of dynamically changing spatial dependencies among roads and temporal patterns over long input sequences. This paper proposes a novel Spatio-Temporal Graph Attention (STGRAT) that effectively captures the spatio-temporal dynamics in road networks. The features of our approach mainly include spatial attention, temporal attention, and spatial sentinel vectors. The spatial attention takes the graph structure information (e.g., distance between roads) and dynamically adjusts spatial correlation based on road states. The temporal attention is responsible for capturing traffic speed changes, while the sentinel vectors allow the model to retrieve new features from spatially correlated nodes or preserve existing features. The experimental results show that STGRAT outperforms existing models, especially in difficult conditions where traffic speeds rapidly change (e.g., rush hours). We additionally provide a qualitative study to analyze when and where STGRAT mainly attended to make accurate predictions during a rush-hour time.

Motivation & Objective

• To address the challenge of predicting traffic speed in complex road networks with dynamic spatial dependencies and abrupt changes.
• To model long-term temporal patterns and spatial correlations that vary with traffic conditions.
• To improve forecasting accuracy in high-variability scenarios such as rush hours, where existing models underperform.
• To enable the model to selectively attend to relevant spatial neighbors or preserve critical features using sentinel vectors.

Proposed method

• Spatial attention dynamically adjusts spatial correlations based on road states and graph structure, such as physical distances between roads.
• Temporal attention captures evolving traffic speed patterns over long input sequences, focusing on relevant temporal dynamics.
• Spatial sentinel vectors allow the model to either retrieve features from spatially correlated nodes or retain existing features, enhancing representation flexibility.
• The model integrates graph convolutional operations with multi-head attention mechanisms to jointly model spatial and temporal dependencies.
• Attention weights are computed using query, key, and value matrices derived from node embeddings, enabling adaptive feature aggregation.
• The architecture is trained end-to-end using a reconstruction loss on historical traffic speed sequences.

Experimental results

Research questions

• RQ1 How can a deep learning model effectively capture dynamically changing spatial dependencies in road networks for traffic forecasting?
• RQ2 To what extent does incorporating temporal attention improve prediction accuracy on long input sequences?
• RQ3 Can sentinel vectors enhance model performance by enabling selective feature retrieval or preservation in spatially correlated regions?
• RQ4 How does STGRAT perform under high-variability traffic conditions such as rush hours compared to existing methods?
• RQ5 Which spatial and temporal regions does STGRAT attend to during critical prediction periods, and how does attention distribution relate to prediction accuracy?

Key findings

• STGRAT outperforms existing state-of-the-art models in traffic speed forecasting, particularly during periods of rapid speed changes such as rush hours.
• The model achieves superior performance on benchmark datasets, demonstrating robustness to abrupt traffic fluctuations.
• Qualitative analysis reveals that STGRAT focuses attention on key road segments and temporally relevant time windows during rush hour, aligning with real traffic dynamics.
• The use of spatial sentinel vectors improves feature representation by allowing adaptive fusion of information from correlated nodes.
• Temporal attention mechanisms effectively capture long-range temporal dependencies, contributing to improved prediction stability over extended sequences.
• The attention mechanism highlights critical junctions and corridors, providing interpretable insights into the model's decision-making process.