[Paper Review] DetectorNet: Transformer-enhanced Spatial Temporal Graph Neural Network for Traffic Prediction

He Li, Shiyu Zhang+7 more|arXiv (Cornell University)|Oct 19, 2021

Traffic Prediction and Management Techniques5 references4 citations

TL;DR

DetectorNet proposes a Transformer-enhanced spatial-temporal graph neural network that models dynamic spatial and temporal correlations in traffic prediction using a Multi-view Temporal Attention module and a Dynamic Attention module. It achieves state-of-the-art performance on two public datasets, outperforming 11 advanced baselines in MAE, RMSE, and MAPE across multiple prediction horizons.

ABSTRACT

Detectors with high coverage have direct and far-reaching benefits for road users in route planning and avoiding traffic congestion, but utilizing these data presents unique challenges including: the dynamic temporal correlation, and the dynamic spatial correlation caused by changes in road conditions. Although the existing work considers the significance of modeling with spatial-temporal correlation, what it has learned is still a static road network structure, which cannot reflect the dynamic changes of roads, and eventually loses much valuable potential information. To address these challenges, we propose DetectorNet enhanced by Transformer. Differs from previous studies, our model contains a Multi-view Temporal Attention module and a Dynamic Attention module, which focus on the long-distance and short-distance temporal correlation, and dynamic spatial correlation by dynamically updating the learned knowledge respectively, so as to make accurate prediction. In addition, the experimental results on two public datasets and the comparison results of four ablation experiments proves that the performance of DetectorNet is better than the eleven advanced baselines.

Motivation & Objective

• To address the limitations of static road network modeling in existing GNN-based traffic prediction methods.
• To capture dynamic spatial correlations that evolve with changing road conditions.
• To model long- and short-range temporal dependencies through multi-view attention mechanisms.
• To improve multi-step traffic prediction accuracy by integrating dynamic structural knowledge.
• To validate the effectiveness of the proposed model against 11 state-of-the-art baselines.

Proposed method

• Introduces a Multi-view Temporal Attention (MTA) module that captures long-, medium-, and short-range temporal dependencies via separate attention heads.
• Employs a Dynamic Attention (DA) module to learn time-varying spatial relationships by dynamically updating the graph structure based on traffic conditions.
• Uses 1D convolutional layers to project input features into higher-dimensional representations before attention modules.
• Applies graph convolutional networks (GCN) with learnable adjacency matrices to model spatial dependencies in a dynamic manner.
• Integrates self-attention mechanisms to enhance long-range temporal sequence modeling, improving multi-step forecasting.
• Trains the model using Adam optimizer with weight decay and dropout (0.3) to prevent overfitting, and evaluates using MAE, RMSE, and MAPE.

Experimental results

Research questions

• RQ1 Can modeling dynamic spatial correlations improve traffic prediction accuracy compared to static graph structures?
• RQ2 Does multi-view temporal attention enhance long-range and multi-step traffic forecasting performance?
• RQ3 How does the integration of dynamic graph learning and self-attention affect prediction robustness and accuracy?
• RQ4 To what extent does DetectorNet outperform existing SOTA models on real-world traffic datasets?
• RQ5 What is the contribution of each component (MTA, DA, SA) to the overall performance?

Key findings

• DetectorNet achieves the lowest MAE of 3.06 and RMSE of 6.08 on METR-LA for 30-minute prediction, outperforming all 11 baselines.
• On PEMS-BAY, DetectorNet records the lowest MAE of 1.57 and RMSE of 3.54 for 30-minute prediction, surpassing all compared models.
• Ablation studies confirm that removing the Multi-view Temporal Attention (MTA) or Dynamic Attention (DA) modules leads to performance degradation, validating their necessity.
• The model shows significant improvement in long-range multi-step prediction due to the self-attention mechanism, outperforming RNN- and TCN-based models.
• The Dynamic Attention module effectively captures evolving road relationships, reducing error by 0.1–0.3 MAE compared to static GNN baselines.
• DetectorNet maintains strong performance across both short- and long-term predictions, with MAPE values below 4% on PEMS-BAY and below 8.2% on METR-LA.