[Paper Review] Learning Stable Deep Dynamics Models

Gaurav Manek, J. Zico Kolter|arXiv (Cornell University)|Jan 17, 2020

Model Reduction and Neural NetworksPhysics and Astronomy18 references55 citations

TL;DR

The paper presents a method to learn neural network dynamics that are provably globally stable by jointly learning a dynamics model and a convex, positive-definite Lyapunov function, enabling stable modeling of simple and complex dynamics, including video textures.

ABSTRACT

Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs). Despite the predictive power of these systems, it has been difficult to make formal claims about the basic properties of the learned systems. In this paper, we propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. The approach works by jointly learning a dynamics model and Lyapunov function that guarantees non-expansiveness of the dynamics under the learned Lyapunov function. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics, such as video textures, in a fully end-to-end fashion.

Motivation & Objective

• Motivate the need for provably stable learned dynamical systems in robotics, control, and forecasting.
• Propose a method to enforce stability globally by integrating a Lyapunov function into the dynamics model.
• Demonstrate the approach on physical systems (e.g., n-link pendulums) and on learning latent dynamics for video textures.
• Show how the stable dynamics can be integrated with larger architectures like VAEs for end-to-end learning.

Proposed method

• Define a nominal dynamics model f̂ and a positive definite Lyapunov function V.
• Constrain the learned dynamics by projecting f̂ onto the halfspace where ∇V(x)ᵀf(x) ≤ −αV(x).
• Represent V as an input-convex neural network (ICNN) to ensure no local optima and convexity.
• Ensure V(0)=0 and add a small ε||x||² term to enforce strict positive definiteness.
• Use a differentiable, smoothed activation to keep V continuously differentiable.
• Optionally allow a warped input space via an invertible function F to relax strong convexity.

Experimental results

Research questions

• RQ1 Can a neural network dynamics model be trained so that it is globally exponentially stable for all states?
• RQ2 Does jointly learning a Lyapunov function with the dynamics guarantee stability in practice for nonlinear systems?
• RQ3 Can the approach scale to higher-dimensional latent spaces and be integrated into end-to-end architectures like VAEs for complex outputs (e.g., video textures)?
• RQ4 How does the stable dynamics model compare to a naive, unconstrained dynamics model on physical systems such as n-link pendulums?

Key findings

• The proposed f built from the projection of f̂ satisfies ∇V(x)ᵀf(x) ≤ −αV(x) for all x, guaranteeing global exponential stability to the origin.
• V is implemented as an ICNN to ensure convexity and absence of local optima, with 0 as the unique minimum due to a normalization term.
• Experiments show stable learned dynamics for random networks, n-link pendulums, and high-dimensional latent video texture dynamics.
• On the pendulum, the stable model maintains energy-consistent behavior and reduces long-horizon error relative to a simple, unstable model.
• In video texture generation, integrating the stable dynamics with a VAE yields diverse, long-horizon frames whereas a naive model diverges.