[Paper Review] Dynamic Spatiotemporal Graph Neural Network with Tensor Network

Chengcheng Jia, Bo Wu+1 more|arXiv (Cornell University)|Mar 12, 2020

Advanced Graph Neural Networks11 references4 citations

TL;DR

This paper proposes a Dynamic Spatiotemporal Graph Neural Network with Tensor Network (DSTGNN) that constructs a Spatial Tensor Graph (STG) and a Temporal Tensor Graph (TTG) to model dynamic spatial and temporal correlations in time series data. By using Projected Entangled Pair States (PEPS) to jointly optimize both graphs, the method reduces parameters via low-rank learning and achieves state-of-the-art performance on traffic forecasting benchmarks, with a 4.43% MAPE at 15-minute prediction on METR-LA.

ABSTRACT

Dynamic spatial graph construction is a challenge in graph neural network (GNN) for time series data problems. Although some adaptive graphs are conceivable, only a 2D graph is embedded in the network to reflect the current spatial relation, regardless of all the previous situations. In this work, we generate a spatial tensor graph (STG) to collect all the dynamic spatial relations, as well as a temporal tensor graph (TTG) to find the latent pattern along time at each node. These two tensor graphs share the same nodes and edges, which leading us to explore their entangled correlations by Projected Entangled Pair States (PEPS) to optimize the two graphs. We experimentally compare the accuracy and time costing with the state-of-the-art GNN based methods on the public traffic datasets.

Motivation & Objective

• To address the limitation of static spatial graphs in GNNs for time series data by modeling dynamic spatial relationships.
• To capture latent temporal correlations at each node by constructing a node-specific temporal tensor graph (TTG).
• To jointly optimize spatial and temporal graphs using a tensor network to exploit their entangled correlations.
• To reduce model parameters and computational complexity through low-rank tensor decomposition via PEPS.
• To improve time series forecasting performance, especially in dynamic environments like traffic networks.

Proposed method

• Constructs a Spatial Tensor Graph (STG) as a 3D tensor 𝒜 ∈ ℝ^{N×N×T} to encode dynamic spatial relationships across time steps.
• Builds a Temporal Tensor Graph (TTG) as a 3D tensor ℬ ∈ ℝ^{N×T×T} to model temporal dependencies per node using similarity in time-series patterns.
• Applies a Spatial/Temporal Graph Convolution Layer (STGCL) using Tucker decomposition to perform high-order convolution on STG and TTG.
• Uses Projected Entangled Pair States (PEPS) to perform joint optimization of STG and TTG, enabling parameter reduction through low-rank tensor contraction.
• Employs Singular Value Decomposition (SVD) and normalization (ReLU, SoftMax) to refine dynamic graph representations at each time step.
• Integrates PEPS as a preprocessing module to extract shared correlations between STG and TTG while minimizing parameter count.

Experimental results

Research questions

• RQ1 Can a dynamic spatial tensor graph (STG) better capture evolving spatial dependencies in time series compared to static graphs?
• RQ2 Can a node-wise temporal tensor graph (TTG) effectively model latent temporal patterns such as rush hour cycles in traffic data?
• RQ3 Does joint optimization of STG and TTG via PEPS improve model performance while reducing parameter count?
• RQ4 How does the proposed DSTGNN framework compare to state-of-the-art GNN-based models in traffic forecasting tasks?
• RQ5 To what extent does the tensor network-based PEPS mechanism enhance feature representation and generalization in spatiotemporal modeling?

Key findings

• The full DSTGNN model with STG, TTG, and PEPS achieves a 4.43% MAPE at 15-minute prediction on METR-LA, outperforming all baselines.
• Incorporating TTG alone (without PEPS) improves performance over STG-only models, reducing MAPE from 8.79% to 8.10% at 60 minutes.
• Adding PEPS to STG+TTG reduces MAPE from 8.29% to 5.49% at 60 minutes, demonstrating the effectiveness of joint optimization.
• The STG-only variant of DSTGNN performs worse than STGCN and Graph WaveNet, indicating the necessity of temporal modeling.
• The PEPS module reduces model parameters by exploiting low-rank correlations between STG and TTG, enhancing efficiency.
• The model achieves a 2.75 MAE at 15-minute prediction, surpassing Graph WaveNet (2.69 MAE) and ARIMA (3.99 MAE), confirming state-of-the-art performance.