[Paper Review] Graph WaveNet for Deep Spatial-Temporal Graph Modeling

Zonghan Wu, Shirui Pan+3 more|arXiv (Cornell University)|May 31, 2019

Advanced Graph Neural Networks26 references70 citations

TL;DR

Graph WaveNet introduces a self-adaptive adjacency matrix and stacked dilated causal convolutions to model spatial-temporal graphs, achieving state-of-the-art traffic forecasting with efficient end-to-end training.

ABSTRACT

Spatial-temporal graph modeling is an important task to analyze the spatial relations and temporal trends of components in a system. Existing approaches mostly capture the spatial dependency on a fixed graph structure, assuming that the underlying relation between entities is pre-determined. However, the explicit graph structure (relation) does not necessarily reflect the true dependency and genuine relation may be missing due to the incomplete connections in the data. Furthermore, existing methods are ineffective to capture the temporal trends as the RNNs or CNNs employed in these methods cannot capture long-range temporal sequences. To overcome these limitations, we propose in this paper a novel graph neural network architecture, Graph WaveNet, for spatial-temporal graph modeling. By developing a novel adaptive dependency matrix and learn it through node embedding, our model can precisely capture the hidden spatial dependency in the data. With a stacked dilated 1D convolution component whose receptive field grows exponentially as the number of layers increases, Graph WaveNet is able to handle very long sequences. These two components are integrated seamlessly in a unified framework and the whole framework is learned in an end-to-end manner. Experimental results on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate the superior performance of our algorithm.

Motivation & Objective

• Address the limitations of fixed graph structures in spatial-temporal modeling.
• Enable learning hidden spatial dependencies directly from data.
• Capture long-range temporal dependencies efficiently without RNNs.
• Integrate graph convolution with dilated causal convolutions in an end-to-end framework.

Proposed method

• Introduce a graph convolution layer with a self-adaptive adjacency matrix learned from node embeddings.
• Combine diffusion-based and self-adaptive graph convolutions to model spatial dependencies.
• Employ stacked dilated causal convolutions (Gated TCN) to capture long-range temporal patterns.
• Use a unified end-to-end framework where spatial-temporal layers are stacked with residual and skip connections.
• Train the model to predict multi-step ahead outputs jointly to avoid train/infer mismatch.
• Output predictions for all T steps in one forward pass rather than recursively.

Experimental results

Research questions

• RQ1 Can a self-adaptive adjacency matrix uncover hidden spatial dependencies beyond a fixed graph?
• RQ2 Does integrating adaptive spatial connections with dilated temporal convolutions improve long-horizon forecasting?
• RQ3 How does Graph WaveNet compare to existing spatial-temporal models on traffic datasets in terms of accuracy and efficiency?

Key findings

Dataset	Model	15 min MAE	15 min RMSE	15 min MAPE	30 min MAE	30 min RMSE	30 min MAPE	60 min MAE	60 min RMSE	60 min MAPE
METR-LA	ARIMA	3.99	8.21	9.60%	5.15	10.45	12.70%	6.90	13.23	17.40%
METR-LA	FC-LSTM	3.44	6.30	9.60%	3.77	7.23	10.90%	4.37	8.69	13.20%
METR-LA	WaveNet	2.99	5.89	8.04%	3.59	7.28	10.25%	4.45	8.93	13.62%
METR-LA	DCRNN	2.77	5.38	7.30%	3.15	6.45	8.80%	3.60	7.60	10.50%
METR-LA	GGRU	2.71	5.24	6.99%	3.12	6.36	8.56%	3.64	7.65	10.62%
METR-LA	STGCN	2.88	5.74	7.62%	3.47	7.24	9.57%	4.59	9.40	12.70%
METR-LA	Graph WaveNet	2.69	5.15	6.90%	3.07	6.22	8.37%	3.53	7.37	10.01%
PEMS-BAY	ARIMA	1.62	3.30	3.50%	2.33	4.76	5.40%	3.38	6.50	8.30%
PEMS-BAY	FC-LSTM	2.05	4.19	4.80%	2.20	4.55	5.20%	2.37	4.96	5.70%
PEMS-BAY	WaveNet	1.39	3.01	2.91%	1.83	4.21	4.16%	2.35	5.43	5.87%
PEMS-BAY	DCRNN	1.38	2.95	2.90%	1.74	3.97	3.90%	2.07	4.74	4.90%
PEMS-BAY	STGCN	1.36	2.96	2.90%	1.81	4.27	4.17%	2.49	5.69	5.79%
PEMS-BAY	Graph WaveNet	1.30	2.74	2.73%	1.63	3.70	3.67%	1.95	4.52	4.63%

• Graph WaveNet achieves state-of-the-art results on METR-LA and PEMS-BAY for 15, 30, and 60 minute horizons.
• The forward-backward-adaptive adjacency configuration yields the best performance across both datasets.
• The adaptive-only configuration performs well when graph structure is unavailable, approaching forward-only performance.
• Graph WaveNet is faster at inference and substantially faster than DCRNN during training, while remaining more accurate than many baselines.
• Learning a self-adaptive adjacency matrix reveals influential nodes and uncovers hidden spatial dependencies.
• The model remains stable and provides improved multi-step forecasts compared to WaveNet and STGCN.