[Paper Review] Neural Stochastic Control

Jingdong Zhang, Qunxi Zhu+1 more|arXiv (Cornell University)|Sep 15, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes two neural stochastic control frameworks—exponential stabilizer (ES) and asymptotic stabilizer (AS)—that leverage stochastic Lyapunov theory to stabilize dynamical systems governed by stochastic differential equations (SDEs). The ES ensures exponential convergence but requires longer training, while the AS enables faster training with asymptotic stability, offering complementary advantages in convergence speed and computational efficiency across physical systems like the Van der Pol oscillator, coupled Stuart-Landau systems, and cell fate dynamics.

ABSTRACT

Control problems are always challenging since they arise from the real-world systems where stochasticity and randomness are of ubiquitous presence. This naturally and urgently calls for developing efficient neural control policies for stabilizing not only the deterministic equations but the stochastic systems as well. Here, in order to meet this paramount call, we propose two types of controllers, viz., the exponential stabilizer (ES) based on the stochastic Lyapunov theory and the asymptotic stabilizer (AS) based on the stochastic asymptotic stability theory. The ES can render the controlled systems exponentially convergent but it requires a long computational time; conversely, the AS makes the training much faster but it can only assure the asymptotic (not the exponential) attractiveness of the control targets. These two stochastic controllers thus are complementary in applications. We also investigate rigorously the linear controller and the proposed neural stochastic controllers in both convergence time and energy cost and numerically compare them in these two indexes. More significantly, we use several representative physical systems to illustrate the usefulness of the proposed controllers in stabilization of dynamical systems.

Motivation & Objective

• To address the challenge of stabilizing stochastic dynamical systems with inherent randomness and uncertainty, especially when system models are partially unknown.
• To develop neural network-based control policies that can stabilize nonlinear and stochastic systems beyond the reach of classical linearization-based methods.
• To provide a theoretical and empirical comparison between neural stochastic controllers and classical linear control in terms of convergence time and energy cost.
• To extend the framework to data-driven settings using Neural ODEs for model-free control in systems like cell fate dynamics.
• To demonstrate the practical efficacy of the proposed controllers on representative physical systems with real-world relevance.

Proposed method

• Proposes two neural stochastic control frameworks: the exponential stabilizer (ES) based on stochastic Lyapunov theory for exponential convergence, and the asymptotic stabilizer (AS) based on stochastic asymptotic stability theory for faster training.
• Employs fully connected feedforward neural networks (FNNs) to learn control functions $\bm{u}_f$ and $\bm{u}_g$ that modify drift and diffusion terms in SDEs: $\mathrm{d}\bm{x} = [f(\bm{x}) + \bm{u}_f(\bm{x})]\mathrm{d}t + [g(\bm{x}) + \bm{u}_g(\bm{x})]\mathrm{d}B_t$.
• Uses input-convex neural networks (ICNNs) and quadratic-form neural networks to ensure the neural Lyapunov function is positive definite and convex, enabling stability certification.
• Integrates data reconstruction via Neural ODEs (NODEs) to learn the underlying vector field $\hat{f}$ from time series data, enabling model-free control in systems with unknown dynamics.
• Employs theoretical analysis to estimate convergence time and energy cost for both linear and neural stochastic controllers, enabling quantitative comparison.
• Validates the framework on benchmark systems: Van der Pol oscillator, coupled Stuart-Landau equations, and cell fate dynamics, using both simulation and trajectory tracking.

Experimental results

Research questions

• RQ1 Can neural stochastic control outperform classical linear control in convergence time and energy cost for stochastic dynamical systems?
• RQ2 How do the ES and AS frameworks compare in terms of convergence speed, training time, and stability guarantees for SDEs?
• RQ3 Can the proposed neural stochastic control be extended to model-free settings using time-series data and Neural ODEs?
• RQ4 What is the theoretical relationship between the control policy, Lyapunov function, and convergence rate in stochastic systems?
• RQ5 How effective is the framework in stabilizing complex physical systems such as synchronization in coupled oscillators and cell fate decisions?

Key findings

• The neural stochastic control frameworks—ES and AS—achieve better convergence performance than classical linear control in both convergence time and energy cost across tested systems.
• The ES ensures exponential stability but requires significantly longer training time, while the AS enables faster training at the cost of only asymptotic stability.
• In the coupled Stuart-Landau system, the AS successfully stabilizes the synchronization manifold, demonstrating effectiveness in complex networked dynamics.
• For the cell fate dynamics system, the proposed method successfully redirects trajectories from stable fates (P2, P3) to the unstable critical state (P1) via pinning control on a high-degree node.
• The data-driven extension using Neural ODEs successfully reconstructs the vector field and enables effective control from time-series data alone, validating the model-free applicability.
• Theoretical analysis confirms that neural stochastic controllers can achieve superior convergence and energy efficiency compared to linear control, especially in nonlinear and stochastic regimes.