[Paper Review] Calibrating multi-dimensional complex ODE from noisy data via deep neural networks

Kexuan Li, Fangfang Wang+3 more|arXiv (Cornell University)|Jun 7, 2021

Model Reduction and Neural Networks30 references4 citations

TL;DR

This paper proposes a two-stage deep learning framework to calibrate high-dimensional nonlinear ODEs from noisy, irregularly spaced data. First, boundary kernel smoothing de-noises the data and estimates derivatives; second, a ReLU-activated, sparsely connected deep neural network models the ODE's right-hand side, achieving consistent estimation under a modular structure assumption, with theoretical convergence rates independent of dimensionality and validated in simulations and a U.S. COVID-19 case study.

ABSTRACT

Ordinary differential equations (ODEs) are widely used to model complex dynamics that arises in biology, chemistry, engineering, finance, physics, etc. Calibration of a complicated ODE system using noisy data is generally very difficult. In this work, we propose a two-stage nonparametric approach to address this problem. We first extract the de-noised data and their higher order derivatives using boundary kernel method, and then feed them into a sparsely connected deep neural network with ReLU activation function. Our method is able to recover the ODE system without being subject to the curse of dimensionality and complicated ODE structure. When the ODE possesses a general modular structure, with each modular component involving only a few input variables, and the network architecture is properly chosen, our method is proven to be consistent. Theoretical properties are corroborated by an extensive simulation study that demonstrates the validity and effectiveness of the proposed method. Finally, we use our method to simultaneously characterize the growth rate of Covid-19 infection cases from 50 states of the USA.

Motivation & Objective

• To address the challenge of calibrating multi-dimensional, nonlinear ODEs when data are noisy, irregularly spaced, and asynchronously observed.
• To develop a nonparametric estimation method that avoids the curse of dimensionality in high-dimensional ODE systems.
• To provide theoretical consistency and convergence rate guarantees for deep neural network estimation of ODEs under a modular function structure.
• To enable joint estimation of ODE components across interdependent systems, such as geographically adjacent states in epidemiological modeling.
• To enhance interpretability and accuracy by integrating feature selection into the neural network architecture.

Proposed method

• Stage 1 employs boundary kernel methods to estimate the true state variables and their higher-order derivatives from noisy, irregularly spaced observations.
• Stage 2 uses a sparsely connected feedforward deep neural network with ReLU activation to approximate the ODE's right-hand side function, f₀.
• The method assumes a general modular structure where each component of f₀ depends only on a small subset of input variables (e.g., x, x⁽¹⁾, ..., x⁽ᵛ⁻¹⁾).
• Theoretical consistency is established under mild smoothness and modular structure assumptions, with convergence rates depending on network depth, width, and function smoothness, not on the overall dimension d.
• A feature selection method based on Lemhadri et al. (2021) is applied to identify the most relevant input variables for each ODE component.
• The framework is validated through extensive simulations and applied to estimate the growth rates of daily COVID-19 cases across U.S. states, incorporating spatial dependencies.

Experimental results

Research questions

• RQ1 Can a deep neural network consistently estimate the right-hand side of a high-dimensional, nonlinear ODE system when data are noisy and irregularly sampled?
• RQ2 Does the proposed method avoid the curse of dimensionality in ODE calibration, even when the true function is complex and high-dimensional?
• RQ3 How does joint estimation across interdependent systems (e.g., neighboring U.S. states) improve accuracy compared to individual estimation?
• RQ4 What is the theoretical convergence rate of the neural network estimator under a modular ODE structure?
• RQ5 Can feature selection within the neural network effectively identify the most relevant variables influencing each ODE component?

Key findings

• The proposed method achieves consistent estimation of the ODE's right-hand side function f₀ under a general modular structure, with convergence rates independent of the system's dimension d.
• Theoretical analysis shows that the convergence rate depends only on the smoothness of f₀, the depth and width of the neural network, and the noise level, not on the number of state variables.
• In simulations, the method outperforms traditional nonparametric derivative estimation, especially in high-dimensional and noisy settings.
• The joint estimation of state-level COVID-19 growth rates using the proposed method produced more consistent and geographically coherent results than individual estimation, with neighboring states showing synchronized trends.
• Feature selection identified New York, Rhode Island, and Connecticut as the most influential states for Massachusetts, reflecting plausible regional transmission dynamics.
• The method successfully predicted that peak daily new cases would occur about one month after the growth rate reached its maximum, demonstrating predictive utility.