[Paper Review] Multivariate Time Series Forecasting Based on Causal Inference with Transfer Entropy and Graph Neural Network.

Haoyan Xu, Yida Huang+3 more|arXiv (Cornell University)|May 3, 2020

Time Series Analysis and Forecasting10 references17 citations

TL;DR

This paper proposes TEGNN, an end-to-end deep learning model that integrates transfer entropy (TE) to model causal relationships among multivariate time series (MTS) variables and uses graph neural networks (GNNs) for forecasting. By constructing a TE-based graph to represent causal dependencies and applying multi-scale CNNs for temporal feature extraction, TEGNN achieves state-of-the-art performance on three real-world MTS benchmark datasets.

ABSTRACT

Multivariate time series (MTS) forecasting is an important problem in many fields. Accurate forecasting results can effectively help decision-making. To date, many MTS forecasting methods have been proposed and widely applied. However, these methods assume that the predicted value of a single variable is affected by all other variables, which ignores the causal relationship among variables. To address the above issue, a novel end-to-end deep learning model, termed graph neural network with transfer entropy (TEGNN) is proposed in this paper. To characterize the causal information among variables, the transfer entropy (TE) graph is introduced in our model, where each variable is regarded as a graph node and each edge represents the casual relationship between variables. In addition, convolutional neural network (CNN) filters with different perception scales are used for time series feature extraction, which is used to generate the feature of each node. Finally, graph neural network (GNN) is adopted to tackle the forecasting problem of graph structure generated by MTS. Three benchmark datasets from the real world are used to evaluate the proposed TEGNN and the comprehensive experiments show that the proposed method achieves state-of-the-art results in MTS forecasting task.

Motivation & Objective

• To address the limitation in existing MTS forecasting methods that assume all variables influence each other without considering causal relationships.
• To model the true causal dependencies among multivariate time series variables using transfer entropy (TE) to construct a causal graph.
• To develop an end-to-end deep learning framework that leverages both temporal features and causal graph structure for improved forecasting accuracy.
• To evaluate the proposed method on real-world MTS datasets and demonstrate its superiority over existing state-of-the-art approaches.

Proposed method

• Construct a transfer entropy (TE) graph where each variable is a node and each edge represents a causal influence from one variable to another.
• Use convolutional neural network (CNN) filters with multiple receptive fields to extract multi-scale temporal features from each time series variable.
• Represent each node in the graph with its extracted temporal features, forming a node feature matrix for the GNN.
• Apply a graph neural network (GNN) to propagate and aggregate information across the TE-based graph structure for forecasting.
• Train the entire model end-to-end using a supervised loss function to minimize forecasting error on future time steps.
• Integrate causal structure (via TE) and temporal dynamics (via multi-scale CNNs) into a unified deep learning framework for MTS forecasting.

Experimental results

Research questions

• RQ1 Can modeling causal relationships among multivariate time series variables improve forecasting accuracy compared to methods that assume symmetric or full interdependence?
• RQ2 How effective is transfer entropy in identifying meaningful causal dependencies between time series variables in real-world datasets?
• RQ3 To what extent does integrating a TE-based graph structure enhance the performance of graph neural networks in MTS forecasting?
• RQ4 Does the proposed TEGNN model outperform existing state-of-the-art methods on diverse real-world MTS benchmarks?

Key findings

• The proposed TEGNN model achieves state-of-the-art performance on three real-world multivariate time series forecasting benchmarks.
• Incorporating transfer entropy to model causal relationships leads to more accurate forecasting than methods that ignore causal structure.
• The use of multi-scale CNN filters enhances temporal feature representation, contributing to improved model generalization.
• The GNN effectively leverages the causal graph structure to propagate relevant information across variables, improving forecasting performance.