[Paper Review] Learning Dynamical Systems from Partial Observations

Ibrahim Ayed, Emmanuel de Bézenac+3 more|arXiv (Cornell University)|Feb 26, 2019

Model Reduction and Neural NetworksPhysics and Astronomy38 references54 citations

TL;DR

The paper proposes a data-driven framework that learns a time-varying differential equation F_theta(X) from partial observations Y by embedding it in a continuous-time ODE and using adjoint-based training to forecast and uncover hidden states, demonstrated on fluid and ocean simulations.

ABSTRACT

We consider the problem of forecasting complex, nonlinear space-time processes when observations provide only partial information of on the system's state. We propose a natural data-driven framework, where the system's dynamics are modelled by an unknown time-varying differential equation, and the evolution term is estimated from the data, using a neural network. Any future state can then be computed by placing the associated differential equation in an ODE solver. We first evaluate our approach on shallow water and Euler simulations. We find that our method not only demonstrates high quality long-term forecasts, but also learns to produce hidden states closely resembling the true states of the system, without direct supervision on the latter. Additional experiments conducted on challenging, state of the art ocean simulations further validate our findings, while exhibiting notable improvements over classical baselines.

Motivation & Objective

• Forecast complex spatio-temporal processes from partial observations

Proposed method

• Model X_t with dX_t/dt = F_theta(X_t) and observations Y_t = H(X_t)
• Learn F_theta and initial condition g_theta from data via a continuous-time optimal control formulation
• Use an adjoint state method to compute gradients and train theta by backpropagation through time
• Discretize forward and backward dynamics (Euler) and train with gradient descent on J = ∫ ||Y_t - H(X_t)||^2 dt
• Handle partial initialization when full state is not observed (via an encoder g_theta or E_theta)
• Evaluate on simulated and realistic datasets including Glorys2v4 to compare with baselines

Experimental results

Research questions

• RQ1 Can a neural network learn the underlying continuous-time dynamics from partial observations?
• RQ2 Can the model forecast observations long-horizon while inferring the hidden state without direct supervision?
• RQ3 How does the method perform with irregular or partial initial information and in realistic, high-dimensional settings?

Key findings

• The method achieves high-quality long-term forecasts for shallow water and Euler equations under partial observation settings.
• Hidden states inferred by the model closely resemble the true hidden dynamics without direct supervision.
• On Glorys2v4, the proposed method outperforms baselines in predicting observations for horizons K=5 and K=10 (0.124 vs 0.145; 0.231 vs 0.250).
• With estimation variant, mean-squared error further improves to 0.113 (K=5) and 0.209 (K=10).
• Hidden-state evaluation shows higher cosine similarity for the proposed method (0.782 at K=5, 0.678 at K=10) than baselines (PKnI: 0.448, 0.371; ConvLSTM: not reported).
• The approach outperforms ConvLSTM baselines across reported metrics on Glorys2v4.