[Paper Review] Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems

Andreas Schlaginhaufen, Philippe Wenk+2 more|arXiv (Cornell University)|Oct 27, 2021

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes Neural Delay Differential Equations (NDDEs) with a novel Lyapunov-Razumikhin regularization to learn stable, partially observed or delayed dynamical systems. By augmenting states with their history and enforcing stability via a neural network Lyapunov-Razumikhin function, the method ensures exponential convergence to equilibrium even under sparse, noisy observations and input delays, enabling robust system identification and stabilizing feedback policy learning.

ABSTRACT

Learning how complex dynamical systems evolve over time is a key challenge in system identification. For safety critical systems, it is often crucial that the learned model is guaranteed to converge to some equilibrium point. To this end, neural ODEs regularized with neural Lyapunov functions are a promising approach when states are fully observed. For practical applications however, partial observations are the norm. As we will demonstrate, initialization of unobserved augmented states can become a key problem for neural ODEs. To alleviate this issue, we propose to augment the system's state with its history. Inspired by state augmentation in discrete-time systems, we thus obtain neural delay differential equations. Based on classical time delay stability analysis, we then show how to ensure stability of the learned models, and theoretically analyze our approach. Our experiments demonstrate its applicability to stable system identification of partially observed systems and learning a stabilizing feedback policy in delayed feedback control.

Motivation & Objective

• To address the instability and poor generalization of standard Neural ODEs when applied to partially observed or delayed dynamical systems.
• To overcome the critical challenge of initializing unobserved augmented states in Augmented Neural ODEs (ANODEs) for dynamical systems.
• To extend neural Lyapunov function regularization from ODEs to time-delay systems using a Lyapunov-Razumikhin approach.
• To enable stable system identification and learning of stabilizing feedback policies in systems with input delays.
• To provide theoretical stability guarantees and practical implementation for NDDEs with a novel regularization scheme.

Proposed method

• Augment the system state with its past history to model non-Markovian dynamics, transforming standard Neural ODEs into Neural Delay Differential Equations (NDDEs).
• Introduce a neural network-based Lyapunov-Razumikhin function to enforce exponential stability of the NDDE model.
• Design a regularization loss based on the Lyapunov-Razumikhin theorem, ensuring that the derivative of the Lyapunov function along trajectories is negative definite within a delay-invariant domain.
• Use a discretized numerical scheme to handle the infinite-dimensional state space of time-delay systems, enabling training via backpropagation.
• Train the NDDE with a combined loss: data reconstruction loss and the proposed Lyapunov-Razumikhin regularization loss.
• Apply the framework to learn stabilizing feedback policies for delayed control systems, using initial LQR gains as warm-starts.

Experimental results

Research questions

• RQ1 Can NDDEs with history-augmented states effectively model partially observed or delayed dynamical systems where standard Neural ODEs fail?
• RQ2 Does the proposed Lyapunov-Razumikhin regularization ensure exponential stability of the learned NDDE model on unseen trajectories?
• RQ3 How does the NDDE approach compare to ANODEs in terms of stability and generalization under partial observations and noise?
• RQ4 Can the framework successfully learn stabilizing feedback policies for delayed control systems, even when the initial policy is unstable?
• RQ5 What is the impact of hyperparameters α and q on the convergence rate and stability of the learned dynamics?

Key findings

• The NDDE model with history augmentation outperforms standard ANODEs in stability and generalization on partially observed systems, avoiding the need to estimate unobserved initial states.
• The proposed Lyapunov-Razumikhin regularization ensures exponential stability of the learned dynamics, with the LRF loss being zero along new test trajectories, indicating valid stability certification.
• For the delayed inverted pendulum, the learned feedback policy stabilized the system, and the decay rate could be controlled via hyperparameters α and q.
• In the cartpole experiment with a 0.05s input delay, the method successfully learned a stabilizing feedback policy where the initial LQR controller was unstable.
• The method achieved stable system identification across multiple test trajectories, with median test loss within the 0.05–0.95 quantile range, demonstrating robustness to noise and initialization.
• The code and theoretical analysis are publicly available, enabling reproducibility and extension to other delayed or partially observed dynamical systems.