[Paper Review] GMAN: A Graph Multi-Attention Network for Traffic Prediction

Chuanpan Zheng, Xiaoliang Fan+2 more|arXiv (Cornell University)|Nov 11, 2019

Traffic Prediction and Management Techniques34 references87 citations

TL;DR

GMAN introduces a graph multi-attention encoder-decoder model with spatial, temporal, and transform attention to predict multi-step traffic conditions on road networks, improving long-term accuracy and fault tolerance.

ABSTRACT

Long-term traffic prediction is highly challenging due to the complexity of traffic systems and the constantly changing nature of many impacting factors. In this paper, we focus on the spatio-temporal factors, and propose a graph multi-attention network (GMAN) to predict traffic conditions for time steps ahead at different locations on a road network graph. GMAN adapts an encoder-decoder architecture, where both the encoder and the decoder consist of multiple spatio-temporal attention blocks to model the impact of the spatio-temporal factors on traffic conditions. The encoder encodes the input traffic features and the decoder predicts the output sequence. Between the encoder and the decoder, a transform attention layer is applied to convert the encoded traffic features to generate the sequence representations of future time steps as the input of the decoder. The transform attention mechanism models the direct relationships between historical and future time steps that helps to alleviate the error propagation problem among prediction time steps. Experimental results on two real-world traffic prediction tasks (i.e., traffic volume prediction and traffic speed prediction) demonstrate the superiority of GMAN. In particular, in the 1 hour ahead prediction, GMAN outperforms state-of-the-art methods by up to 4% improvement in MAE measure. The source code is available at https://github.com/zhengchuanpan/GMAN.

Motivation & Objective

• Motivate long-term traffic prediction under dynamic spatio-temporal conditions on road networks.
• Develop an encoder-decoder architecture with spatio-temporal attention blocks to model complex correlations.
• Incorporate a transform attention layer to alleviate error propagation across prediction steps.
• Integrate spatial and temporal embeddings to capture graph structure and time context.
• Evaluate GMAN on real-world traffic volume and speed prediction datasets and compare with baselines.

Proposed method

• Use an encoder-decoder architecture with L ST-Attention blocks in both encoder and decoder.
• Incorporate spatio-temporal embedding (STE) combining node2vec-based spatial embeddings with time embeddings.
• Implement ST-Attention block with spatial attention, temporal attention, and gated fusion to merge both signals.
• Introduce a transform attention layer between encoder and decoder to model direct relationships between historical and future steps.
• Apply group spatial attention (intra-group and inter-group) to reduce quadratic attention complexity for large graphs.
• Train end-to-end with MAE loss and use adjacency constructed from road-network distances with exponential sparsification.

Experimental results

Research questions

• RQ1 How can we model dynamic spatial correlations and non-linear temporal dependencies in traffic data over longer horizons?
• RQ2 Does an encoder-decoder with spatio-temporal attention and a transform attention layer improve long-term prediction accuracy compared to existing graph-based models?
• RQ3 Can a gated fusion of spatial and temporal information better capture complex spatio-temporal dependencies in traffic networks?
• RQ4 What is the fault tolerance of GMAN to missing or corrupted historical observations in long-horizon predictions?

Key findings

• GMAN achieves state-of-the-art performance on two real-world datasets (Xiamen for traffic volume and PeMS for traffic speed).
• In 1-hour-ahead predictions, GMAN improves MAE over baselines by up to 4% and demonstrates superior long-horizon performance.
• GMAN shows higher fault tolerance when historical data is partially missing compared to baseline methods.
• Ablation studies show spatial attention, temporal attention, gated fusion, and transform attention each contribute to performance, with transform attention notably easing error propagation.
• GMAN maintains competitive training and inference times relative to Graph WaveNet and faster than recurrent baselines like DCRNN.