[Paper Review] Dynamic Graph Learning-Neural Network for Multivariate Time Series Modeling

Zhuoling Li, Gaowei Zhang+2 more|arXiv (Cornell University)|Dec 6, 2021

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes SDGL, a dynamic graph learning-neural network that models multivariate time series by learning both static (long-term) and dynamic (short-term) graph structures from data. By fusing node features with learned static graphs via a gating mechanism and using a multi-headed adjacency module, SDGL captures evolving spatio-temporal dependencies, achieving state-of-the-art performance on traffic and benchmark time series datasets.

ABSTRACT

Multivariate time series forecasting is a challenging task because the data involves a mixture of long- and short-term patterns, with dynamic spatio-temporal dependencies among variables. Existing graph neural networks (GNN) typically model multivariate relationships with a pre-defined spatial graph or learned fixed adjacency graph. It limits the application of GNN and fails to handle the above challenges. In this paper, we propose a novel framework, namely static- and dynamic-graph learning-neural network (SDGL). The model acquires static and dynamic graph matrices from data to model long- and short-term patterns respectively. Static matric is developed to capture the fixed long-term association pattern via node embeddings, and we leverage graph regularity for controlling the quality of the learned static graph. To capture dynamic dependencies among variables, we propose dynamic graphs learning method to generate time-varying matrices based on changing node features and static node embeddings. And in the method, we integrate the learned static graph information as inductive bias to construct dynamic graphs and local spatio-temporal patterns better. Extensive experiments are conducted on two traffic datasets with extra structural information and four time series datasets, which show that our approach achieves state-of-the-art performance on almost all datasets. If the paper is accepted, I will open the source code on github.

Motivation & Objective

• Address the challenge of modeling mixed long- and short-term patterns in multivariate time series.
• Overcome limitations of fixed or pre-defined graphs in existing GNNs that fail to capture dynamic dependencies.
• Develop a framework that learns both stable long-term relationships and time-varying short-term interactions among variables.
• Integrate inductive bias from static graphs into dynamic graph learning to improve modeling of local spatio-temporal changes.
• Achieve superior forecasting performance without relying on prior graph structures in real-world multivariate time series.

Proposed method

• Learn a static graph matrix from node embeddings to capture fixed long-term dependencies, regularized via graph regularity to ensure structural quality.
• Construct dynamic graphs using time-varying node features and static graph embeddings, enabling time-variant modeling of short-term interactions.
• Employ a gating mechanism to fuse dynamic node inputs with static graph information, enhancing representation learning of evolving dependencies.
• Use a multi-headed adjacency mechanism to efficiently extract and model associations between variables in dynamic graphs.
• Integrate static graph structure as inductive bias in dynamic graph learning, ensuring dynamic graphs fluctuate around long-term patterns.
• Apply graph regularization to stabilize and improve the quality of the learned static graph during training.

Experimental results

Research questions

• RQ1 How can multivariate time series models effectively capture both long-term and short-term temporal patterns?
• RQ2 Can dynamic graph learning improve forecasting accuracy when relationships between variables evolve over time?
• RQ3 How does incorporating static graph structure as inductive bias enhance the learning of dynamic dependencies?
• RQ4 What is the contribution of each component (e.g., dynamic graph learning, information fusion, regularization) to overall model performance?
• RQ5 Can a graph learning framework achieve state-of-the-art results without relying on predefined or handcrafted graph structures?

Key findings

• SDGL achieves state-of-the-art performance on both traffic datasets (PeMSD4 and PeMSD8) and four benchmark time series datasets, outperforming existing methods in most cases.
• The ablation study shows that removing the dynamic graph learning layer leads to a significant performance drop, especially in long-horizon predictions (e.g., 60-minute forecasts).
• The information fusion module (IFM) is critical: performance drops substantially when only dynamic inputs are used, highlighting the importance of integrating long-term patterns.
• Graph regularization improves static graph quality, though SDGL without it still outperforms strong baselines like AGCNR, indicating the framework’s inherent robustness.
• Visualization confirms that dynamic graphs learn short-term patterns close to the predefined graph structure, with similar high-weight edges appearing at different time points.
• The dynamic graph matrices show high similarity over short time spans, confirming the model captures stable short-term dependencies that evolve gradually.