[Paper Review] Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations

Ivan Marisca, Andrea Cini+1 more|arXiv (Cornell University)|May 26, 2022

Functional Brain Connectivity Studies19 citations

TL;DR

This paper proposes SPIN, a novel graph neural network with a sparse spatiotemporal attention mechanism that reconstructs missing data in highly sparse multivariate time series by propagating only observed values through space and time. Unlike autoregressive models, SPIN avoids error propagation and achieves state-of-the-art performance on traffic and air quality benchmarks, especially under extreme sparsity.

ABSTRACT

Modeling multivariate time series as temporal signals over a (possibly dynamic) graph is an effective representational framework that allows for developing models for time series analysis. In fact, discrete sequences of graphs can be processed by autoregressive graph neural networks to recursively learn representations at each discrete point in time and space. Spatiotemporal graphs are often highly sparse, with time series characterized by multiple, concurrent, and long sequences of missing data, e.g., due to the unreliable underlying sensor network. In this context, autoregressive models can be brittle and exhibit unstable learning dynamics. The objective of this paper is, then, to tackle the problem of learning effective models to reconstruct, i.e., impute, missing data points by conditioning the reconstruction only on the available observations. In particular, we propose a novel class of attention-based architectures that, given a set of highly sparse discrete observations, learn a representation for points in time and space by exploiting a spatiotemporal propagation architecture aligned with the imputation task. Representations are trained end-to-end to reconstruct observations w.r.t. the corresponding sensor and its neighboring nodes. Compared to the state of the art, our model handles sparse data without propagating prediction errors or requiring a bidirectional model to encode forward and backward time dependencies. Empirical results on representative benchmarks show the effectiveness of the proposed method.

Motivation & Objective

• Address the challenge of reconstructing missing data in multivariate time series with highly sparse, irregular observations common in real sensor networks.
• Overcome the instability and error propagation issues inherent in autoregressive models that rely on predicted values for subsequent reconstructions.
• Design a model that conditions imputation solely on observed data, avoiding reliance on bidirectional or error-prone autoregressive prediction chains.
• Introduce a spatiotemporal attention mechanism that explicitly accounts for sparsity and propagates only valid observations through the graph structure.
• Develop a self-organizing, task-aligned propagation process that learns representations directly for imputation, improving robustness in sparse settings.

Proposed method

• Propose a sparse spatiotemporal attention mechanism that restricts attention to only observed nodes in both space and time, avoiding attention to missing data points.
• Design a graph neural network architecture where each node's representation is updated via message passing that aggregates information only from observed neighbors across time and space.
• Train the model end-to-end to reconstruct missing observations using a loss function that minimizes reconstruction error on observed data points.
• Introduce a hierarchical attention variant (SPIN-H) to reduce computational complexity while preserving performance, especially in large-scale graphs.
• Align the message-passing process with the imputation objective by ensuring that each node’s representation is informed only by valid, observed data points.
• Use a learnable attention mechanism that dynamically weights contributions from neighboring nodes and time steps based on spatiotemporal proximity and data availability.

Experimental results

Research questions

• RQ1 Can a self-organizing spatiotemporal propagation mechanism reconstruct missing data more robustly than autoregressive models in highly sparse settings?
• RQ2 Does conditioning imputation only on observed data, without relying on predicted values, reduce error accumulation and improve generalization under extreme sparsity?
• RQ3 How does the proposed sparse spatiotemporal attention mechanism compare to standard attention and autoregressive models in handling long, consecutive missing sequences?
• RQ4 To what extent does the hierarchical attention variant (SPIN-H) maintain performance while reducing computational cost?
• RQ5 Can the model generalize across different data distributions and missing data patterns (e.g., point-wise vs. block-wise failures) without retraining?

Key findings

• SPIN outperforms all baselines, including GRIN, SAITS, and Transformers, across all benchmarks and sparsity levels, achieving the lowest MAE on METR-LA, PEMS-BAY, and AQI datasets.
• In the Block missing setting with 15% failure probability (≈96–97% missing rate), SPIN achieved a MAE of 24.34 on AQI, significantly outperforming GRIN (29.03) and SAITS (36.34).
• SPIN-H achieved a MAE of 2.64 on METR-LA under 5% failure probability in the Block missing scenario, outperforming GRIN (3.05) and SAITS (4.73).
• The performance of SPIN deteriorates at a slower rate than baselines as sparsity increases, demonstrating superior robustness to extreme missing data patterns.
• SPIN maintains strong performance across different missing data distributions (point-wise and block-wise), indicating generalization capability beyond specific sparsity patterns.
• The model’s end-to-end training on observed data alone, without error propagation, leads to more stable and reliable learning dynamics in sparse settings.