[Paper Review] Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction

Megan R. Ebers, Jan P. Williams+2 more|arXiv (Cornell University)|Jul 20, 2023

Target Tracking and Data Fusion in Sensor NetworksComputer Science3 citations

TL;DR

This paper introduces a shallow recurrent decoder network (SHRED) that leverages mobile sensor trajectories to reconstruct full spatio-temporal states from sparse, dynamic measurements. By encoding temporal sensor history into a low-dimensional latent space, SHRED achieves superior reconstruction accuracy and generalization—reducing mean-square error variance and enabling rapid adaptation to unseen data—demonstrated across turbulent flows, sea-surface temperature, and human biomechanics.

ABSTRACT

Sensing is one of the most fundamental tasks for the monitoring, forecasting and control of complex, spatio-temporal systems. In many applications, a limited number of sensors are mobile and move with the dynamics, with examples including wearable technology, ocean monitoring buoys, and weather balloons. In these dynamic systems (without regions of statistical-independence), the measurement time history encodes a significant amount of information that can be extracted for critical tasks. Most model-free sensing paradigms aim to map current sparse sensor measurements to the high-dimensional state space, ignoring the time-history all together. Using modern deep learning architectures, we show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic trajectory information can be mapped to full state-space estimates. Indeed, we demonstrate that by leveraging mobile sensor trajectories with shallow recurrent decoder networks, we can train the network (i) to accurately reconstruct the full state space using arbitrary dynamical trajectories of the sensors, (ii) the architecture reduces the variance of the mean-square error of the reconstruction error in comparison with immobile sensors, and (iii) the architecture also allows for rapid generalization (parameterization of dynamics) for data outside the training set. Moreover, the path of the sensor can be chosen arbitrarily, provided training data for the spatial trajectory of the sensor is available. The exceptional performance of the network architecture is demonstrated on three applications: turbulent flows, global sea-surface temperature data, and human movement biomechanics.

Motivation & Objective

• Address the challenge of reconstructing high-dimensional spatio-temporal states when only sparse, mobile sensor data is available.
• Overcome the limitations of traditional model-free sensing methods that ignore temporal history and rely only on current measurements.
• Develop a data-driven architecture that leverages sensor trajectory history to encode global system dynamics for improved state estimation.
• Enable robust generalization to unseen data and arbitrary sensor trajectories without requiring prior knowledge of the underlying dynamics.
• Demonstrate the method’s effectiveness across diverse, high-dimensional, nonlinear systems such as turbulent flows and biomechanical motion.

Proposed method

• Employ a sequence-to-vector long short-term memory (LSTM) network to encode the temporal history of mobile sensor measurements into a low-dimensional latent space.
• Use a shallow decoder network to map the latent representation to a full-state estimate of the high-dimensional system.
• Train the architecture end-to-end using paired data of sensor trajectories and corresponding full-state system data.
• Allow arbitrary sensor trajectories during inference, provided trajectory data is available during training.
• Leverage the spatial and temporal coupling in spatio-temporal systems, where sensor movement samples neighboring and global system features through sequential observations.
• Use simulation data as a surrogate for real high-dimensional state data when ground truth is unavailable, enabling hybrid training.

Figure 1: Summary figure of a shallow recurrent decoder network (SHRED) leveraging mobile sensors to reconstruct full state-space estimates from sparse dynamical trajectories. (Left) Sensor trajectory history encodes global information of the spatio-temporal dynamics of the sparsely measured system.

Experimental results

Research questions

• RQ1 Can a deep learning architecture reconstruct full spatio-temporal states from sparse, mobile sensor measurements by leveraging temporal history?
• RQ2 How does the performance of mobile sensing with SHRED compare to immobile sensors in terms of reconstruction error and variance?
• RQ3 To what extent can the SHRED architecture generalize to data outside the training distribution, particularly for new individuals or unseen dynamics?
• RQ4 Does the sensor trajectory’s path shape the quality of state reconstruction, and can arbitrary trajectories be used effectively?
• RQ5 Can simulation data effectively train the SHRED model when real high-dimensional state data is inaccessible?

Key findings

• SHRED reduces the mean-square error variance of reconstruction compared to immobile sensors, demonstrating improved robustness.
• With a single mobile sensor, SHRED achieved reconstruction errors of 0.121 ± 0.073 degrees in the human biomechanics dataset—9.8× more accurate than a standard dense network (SDN) and 14.8× more accurate than a linear model.
• The architecture enables rapid generalization for out-of-distribution data, such as estimating parameters for a new patient in biomechanics using population-level models.
• SHRED generalizes effectively to arbitrary sensor trajectories, provided training data includes the corresponding trajectory paths.
• The shallow decoder structure requires less training data and is robust to hyperparameter tuning, enabling high performance with minimal configuration.
• The method achieves nonlinear generalization beyond classical low-rank techniques like proper orthogonal decomposition (POD), effectively capturing multiscale physics in the latent space.

Figure 2: (Top) Example flow fields to be reconstructed from the test set. (Bottom) Histograms of the difference between ground truth and reconstruction across all nodes and samples in the test set for immobile sensors (left) and mobile sensors (right). Both distributions are approximately zero mean