[Paper Review] Multivariate Time Series Imputation by Graph Neural Networks

Andrea Cini, Ivan Marisca+1 more|arXiv (Cornell University)|Jul 31, 2021

Functional Brain Connectivity Studies57 references8 citations

TL;DR

This paper proposes GRIL, a novel graph neural network architecture for multivariate time series imputation that leverages spatial-temporal message passing to model nonlinear dependencies across interconnected sensors. It achieves state-of-the-art performance, reducing mean absolute error by over 20% on benchmark datasets compared to existing methods.

ABSTRACT

Dealing with missing values and incomplete time series is a labor-intensive and time-consuming inevitable task when handling data coming from real-world applications. Effective spatio-temporal representations would allow imputation methods to reconstruct missing temporal data by exploiting information coming from sensors at different locations. However, standard methods fall short in capturing the nonlinear time and space dependencies existing within networks of interconnected sensors and do not take full advantage of the available - and often strong - relational information. Notably, most of state-of-the-art imputation methods based on deep learning do not explicitly model relational aspects and, in any case, do not exploit processing frameworks able to adequately represent structured spatio-temporal data. Conversely, graph neural networks have recently surged in popularity as both expressive and scalable tools for processing sequential data with relational inductive biases. In this work, we present the first assessment of graph neural networks in the context of multivariate time series imputation. In particular, we introduce a novel graph neural network architecture, named GRIL, which aims at reconstructing missing data in the different channels of a multivariate time series by learning spatial-temporal representations through message passing. Preliminary empirical results show that our model outperforms state-of-the-art methods in the imputation task on relevant benchmarks with mean absolute error improvements often higher than 20%.

Motivation & Objective

• To address the challenge of missing data in multivariate time series from real-world sensor networks.
• To overcome limitations of existing deep learning imputation methods that fail to model relational and structural dependencies.
• To introduce a graph neural network-based framework capable of capturing nonlinear temporal and spatial correlations in sensor data.
• To evaluate the effectiveness of graph-based representations in reconstructing missing values in multivariate time series.

Proposed method

• GRIL constructs a graph where nodes represent sensors and edges encode spatial relationships between them.
• It employs a message-passing mechanism to propagate and aggregate temporal and spatial information across the graph structure.
• The model learns joint spatial-temporal representations by updating node embeddings through multiple layers of graph convolution.
• It uses a learnable attention mechanism to weigh messages from neighboring nodes, enhancing the model's ability to focus on relevant dependencies.
• The architecture is trained end-to-end using a reconstruction loss to minimize the difference between predicted and actual values in observed time series segments.
• GRIL is designed to handle incomplete multivariate time series by jointly modeling temporal dynamics and relational structure.

Experimental results

Research questions

• RQ1 Can graph neural networks effectively model complex spatial-temporal dependencies in multivariate time series with missing data?
• RQ2 How does GRIL compare to state-of-the-art imputation methods in terms of reconstruction accuracy?
• RQ3 To what extent do relational inductive biases improve imputation performance in sensor network data?
• RQ4 Does message passing in a graph-structured representation lead to better generalization on missing data patterns?

Key findings

• GRIL achieves significant improvements in mean absolute error (MAE) over state-of-the-art methods, with reductions often exceeding 20% on benchmark datasets.
• The model effectively captures nonlinear temporal and spatial dependencies that standard deep learning methods fail to exploit.
• By modeling sensor networks as graphs, GRIL leverages relational information to improve imputation accuracy where traditional methods underperform.
• The use of message passing enables the model to propagate information across interconnected sensors, enhancing reconstruction of missing values.
• Empirical results demonstrate that GRIL generalizes well across diverse multivariate time series with varying missing data patterns.
• The architecture shows strong scalability and expressiveness in handling complex, real-world sensor data with incomplete observations.