[Paper Review] Multivariate Time Series Forecasting with Transfer Entropy Graph

Ziheng Duan, Haoyan Xu+3 more|arXiv (Cornell University)|May 3, 2020

Time Series Analysis and Forecasting5 citations

TL;DR

This paper proposes CauGNN, an end-to-end deep learning model for multivariate time series forecasting that integrates transfer entropy (TE) to construct a causal graph and uses graph neural networks (GNNs) with multi-scale CNNs for feature extraction. The method achieves state-of-the-art performance by explicitly modeling nonlinear causal relationships among variables, outperforming existing models on benchmark datasets.

ABSTRACT

Multivariate time series (MTS) forecasting is an essential problem in many fields. Accurate forecasting results can effectively help decision-making. To date, many MTS forecasting methods have been proposed and widely applied. However, these methods assume that the predicted value of a single variable is affected by all other variables, which ignores the causal relationship among variables. To address the above issue, we propose a novel end-to-end deep learning model, termed graph neural network with Neural Granger Causality (CauGNN) in this paper. To characterize the causal information among variables, we introduce the Neural Granger Causality graph in our model. Each variable is regarded as a graph node, and each edge represents the casual relationship between variables. In addition, convolutional neural network (CNN) filters with different perception scales are used for time series feature extraction, which is used to generate the feature of each node. Finally, Graph Neural Network (GNN) is adopted to tackle the forecasting problem of graph structure generated by MTS. Three benchmark datasets from the real world are used to evaluate the proposed CauGNN. The comprehensive experiments show that the proposed method achieves state-of-the-art results in the MTS forecasting task.

Motivation & Objective

• To address the limitation of existing multivariate time series forecasting models that assume all variables influence each other without considering causal relationships.
• To incorporate nonlinear causal dependencies among time series variables using transfer entropy (TE) as a prior.
• To develop an end-to-end deep learning framework that leverages graph structure and causal information for improved forecasting accuracy.
• To validate the effectiveness of the proposed model on real-world benchmark datasets.

Proposed method

• Construct a Neural Granger Causality graph using transfer entropy (TE) to model nonlinear causal relationships between multivariate time series variables.
• Represent each time series variable as a node in a graph, with edges weighted by pairwise TE values to form an adjacency matrix.
• Apply multi-scale 1D convolutional neural network (CNN) filters to extract temporal features from raw time series data for each node.
• Use graph neural networks (GNNs) to aggregate and propagate features across the causal graph for forecasting.
• Train the entire model end-to-end using backpropagation with loss functions including RAE and MAE.
• Integrate regularization and group Lasso in the variable selection module to identify key causal variables.

Experimental results

Research questions

• RQ1 Can transfer entropy effectively capture nonlinear causal relationships among multivariate time series variables for forecasting?
• RQ2 How does incorporating a TE-based causal graph improve forecasting performance compared to models assuming full connectivity?
• RQ3 What is the contribution of multi-scale CNNs in extracting meaningful temporal features for graph-based forecasting?
• RQ4 How robust is the CauGNN model to hyperparameter choices across diverse datasets and forecasting horizons?
• RQ5 To what extent does removing the CNN or causal graph component degrade model performance?

Key findings

• CauGNN outperforms all state-of-the-art baselines on three real-world benchmark datasets, achieving the best results in terms of RAE, MAE, and CORR metrics.
• The ablation study shows that removing the CNN component (CauGNN-nCNN) leads to a significant performance drop, confirming its importance in feature extraction.
• Replacing the GNN layer with GIN layers (CauGNN-GIN) maintains performance, indicating the framework's compatibility with different GNN architectures.
• Using a fully connected (all-one) adjacency matrix (CauGNN-nCau) instead of the TE-based graph results in worse performance, proving the value of causal priors.
• The model is robust to hyperparameter changes, with stable performance across hidden dimension values from 10 to 100 in GNN layers.
• Multiple CNN filters (three) significantly outperform single-filter or raw input (CauGNN-RF), demonstrating the benefit of multi-receptive-field feature learning.