[Paper Review] Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks

Zonghan Wu, Shirui Pan+4 more|arXiv (Cornell University)|May 24, 2020

Time Series Analysis and Forecasting25 references114 citations

TL;DR

MTGNN automatically learns latent directed relationships among variables and jointly trains graph learning, graph convolution, and temporal convolution modules to forecast multivariate time series end-to-end.

ABSTRACT

Modeling multivariate time series has long been a subject that has attracted researchers from a diverse range of fields including economics, finance, and traffic. A basic assumption behind multivariate time series forecasting is that its variables depend on one another but, upon looking closely, it is fair to say that existing methods fail to fully exploit latent spatial dependencies between pairs of variables. In recent years, meanwhile, graph neural networks (GNNs) have shown high capability in handling relational dependencies. GNNs require well-defined graph structures for information propagation which means they cannot be applied directly for multivariate time series where the dependencies are not known in advance. In this paper, we propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module, into which external knowledge like variable attributes can be easily integrated. A novel mix-hop propagation layer and a dilated inception layer are further proposed to capture the spatial and temporal dependencies within the time series. The graph learning, graph convolution, and temporal convolution modules are jointly learned in an end-to-end framework. Experimental results show that our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets and achieves on-par performance with other approaches on two traffic datasets which provide extra structural information.

Motivation & Objective

• Motivate/address the need to exploit latent interdependencies among time series variables.
• Develop a general graph neural network framework that can handle multivariate time series with or without a predefined graph.
• Learn a sparse, uni-directional graph structure jointly with predictive models in an end-to-end fashion.
• Capture spatial dependencies via graph convolutions and temporal dependencies via temporal convolutions.
• Improve forecasting accuracy over state-of-the-art baselines on benchmark datasets.

Proposed method

• Propose MTGNN with three core components: a graph learning layer that adaptively learns a sparse, uni-directional adjacency matrix; graph convolution modules that fuse spatial information; and temporal convolution modules with a dilated inception design to capture long-range temporal patterns.
• Use a mix-hop propagation layer to aggregate information from multiple neighborhood hops while retaining a portion of the node’s original state to combat over-smoothing.
• Employ a temporal convolution module based on dilated inception layers (kernel sizes 1x2, 1x3, 1x6, 1x7) to capture multi-frequency temporal patterns and long sequences.
• Interleave graph and temporal convolutions, with residual and skip connections to stabilize training and enable end-to-end learning.
• Introduce a curriculum learning strategy that trains on progressively longer forecast horizons and a sub-graph sampling scheme to handle large graphs and reduce memory usage.

Experimental results

Research questions

• RQ1 Can a learnable, sparse, uni-directional graph structure be discovered from multivariate time series lacking explicit graph structure?
• RQ2 Does end-to-end joint learning of graph structure and MTGNN improve forecasting accuracy over methods using predefined graphs or no graph?
• RQ3 How do graph convolution and temporal convolution modules complement each other in capturing spatial-temporal dependencies?
• RQ4 Can curriculum learning and sub-graph sampling stabilize training and enable scaling to larger graphs?
• RQ5 How does MTGNN perform relative to state-of-the-art baselines on standard multivariate time series datasets?

Key findings

• MTGNN outperforms state-of-the-art baselines on 3 of 4 benchmark datasets for single-step forecasting.
• MTGNN achieves on-par performance with other GNN-based approaches on two traffic datasets that provide extra structural information.
• A learnable graph learning layer produces an asymmetric, sparse adjacency matrix suitable for directed relationships.
• The mix-hop propagation layer and dilated inception temporal module effectively capture spatial and temporal dependencies while mitigating over-smoothing and enabling long-range forecasting.