[Paper Review] RODE-Net: Learning Ordinary Differential Equations with Randomness from Data

Junyu Liu, Zichao Long+3 more|arXiv (Cornell University)|Jun 3, 2020

Model Reduction and Neural Networks27 references4 citations

TL;DR

RODE-Net is a deep learning framework that jointly learns the symbolic form of ordinary differential equations (ODEs) and the distribution of their random parameters from data, using a symbolic neural network (SymNet) for system identification and a generative adversarial network (GAN) as a data-driven regularization to improve parameter estimation. The method achieves superior accuracy in both ODE structure recovery and parameter distribution estimation compared to conventional ℓ₁ regularization.

ABSTRACT

Random ordinary differential equations (RODEs), i.e. ODEs with random parameters, are often used to model complex dynamics. Most existing methods to identify unknown governing RODEs from observed data often rely on strong prior knowledge. Extracting the governing equations from data with less prior knowledge remains a great challenge. In this paper, we propose a deep neural network, called RODE-Net, to tackle such challenge by fitting a symbolic expression of the differential equation and the distribution of parameters simultaneously. To train the RODE-Net, we first estimate the parameters of the unknown RODE using the symbolic networks \cite{long2019pde} by solving a set of deterministic inverse problems based on the measured data, and use a generative adversarial network (GAN) to estimate the true distribution of the RODE's parameters. Then, we use the trained GAN as a regularization to further improve the estimation of the ODE's parameters. The two steps are operated alternatively. Numerical results show that the proposed RODE-Net can well estimate the distribution of model parameters using simulated data and can make reliable predictions. It is worth noting that, GAN serves as a data driven regularization in RODE-Net and is more effective than the $\ell_1$ based regularization that is often used in system identifications.

Motivation & Objective

• To address the challenge of identifying random ordinary differential equations (RODEs) from data with minimal prior knowledge on the governing equation structure.
• To estimate both the symbolic form of the ODE and the probability distribution of its random parameters simultaneously.
• To overcome limitations of traditional regularization (e.g., ℓ₁) in high-dimensional inverse problems by introducing a data-driven GAN-based regularization.
• To enable reliable simulation of stochastic trajectories that match the statistical behavior of true RODE-generated data.

Proposed method

• RODE-Net combines SymNet for symbolic system identification to recover the ODE structure from data.
• A separate GAN is trained to estimate the true distribution of the ODE's random parameters using observed data snapshots.
• The GAN acts as a data-driven regularizer during the training of the ODE parameter estimation, improving generalization and robustness.
• Training alternates between estimating ODE parameters via deterministic inverse problems and refining the GAN-based parameter distribution.
• The framework leverages the high expressive power of SymNet to minimize reliance on prior assumptions about the ODE form.
• The GAN regularization is applied to reduce prediction errors and improve the accuracy of estimated ODE coefficients.

Experimental results

Research questions

• RQ1 Can a deep learning framework jointly learn the symbolic form of an ODE and the distribution of its random parameters from limited data?
• RQ2 How does GAN-based regularization compare to traditional ℓ₁ regularization in terms of accuracy and robustness for inverse ODE problems?
• RQ3 To what extent can RODE-Net recover the true ODE structure and parameter distribution with minimal prior knowledge?
• RQ4 Does the use of GAN as a regularizer improve the generalization and prediction accuracy of ODE parameter estimation?

Key findings

• RODE-Net significantly reduces prediction errors compared to ODE-Nets without GAN regularization, with notable improvements in individual ODE instances.
• The mean and standard deviation of the error in ODE coefficient estimation across 500 instances were 2.122 and 3.372 for RODE-Net, compared to 2.093 and 3.352 for ODE-Nets without GAN, indicating consistent improvement.
• The estimated coefficients of primary ODE terms (e.g., $x_1x_2$, $x_2$) were closer to the true values when using GAN regularization, as shown in the coefficient comparison table.
• The GAN-based regularization led to better histogram alignment of prediction errors with the true distribution, especially at later time points (t=40,50).
• RODE-Net successfully simulated trajectories that statistically matched those generated by the true RODE, demonstrating effective distribution learning.
• The framework outperformed ℓ₁ regularization in high-dimensional parameter spaces, suggesting GANs are more effective for complex inverse problems.