[Paper Review] Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression

Zhaozhi Qian, William R. Zame+3 more|arXiv (Cornell University)|Jun 5, 2021

Model Reduction and Neural Networks84 references18 citations

TL;DR

This paper proposes the Latent Hybridisation Model (LHM), a novel framework that integrates expert-designed pharmacological ODEs with neural ODEs to model disease progression under treatment. By combining domain-specific ODEs with learnable latent variables, LHM improves sample efficiency and predictive accuracy—especially in low-data regimes—outperforming prior methods on both synthetic and real-world ICU data from COVID-19 patients.

ABSTRACT

Modeling a system's temporal behaviour in reaction to external stimuli is a fundamental problem in many areas. Pure Machine Learning (ML) approaches often fail in the small sample regime and cannot provide actionable insights beyond predictions. A promising modification has been to incorporate expert domain knowledge into ML models. The application we consider is predicting the progression of disease under medications, where a plethora of domain knowledge is available from pharmacology. Pharmacological models describe the dynamics of carefully-chosen medically meaningful variables in terms of systems of Ordinary Differential Equations (ODEs). However, these models only describe a limited collection of variables, and these variables are often not observable in clinical environments. To close this gap, we propose the latent hybridisation model (LHM) that integrates a system of expert-designed ODEs with machine-learned Neural ODEs to fully describe the dynamics of the system and to link the expert and latent variables to observable quantities. We evaluated LHM on synthetic data as well as real-world intensive care data of COVID-19 patients. LHM consistently outperforms previous works, especially when few training samples are available such as at the beginning of the pandemic.

Motivation & Objective

• To address the limitations of pure ML models in small-data regimes and their lack of interpretability in clinical time-series modeling.
• To bridge the gap between clinically unobservable, biologically meaningful variables in pharmacological ODE models and real-world, noisy, irregularly sampled patient measurements.
• To enhance sample efficiency and predictive accuracy in disease progression modeling by embedding domain-specific ODEs into a neural ODE framework.
• To provide clinicians with interpretable latent variables linked to known pharmacological mechanisms while maintaining high predictive performance.

Proposed method

• LHM integrates a system of expert-designed ODEs (representing biologically meaningful variables) with a neural ODE that models latent dynamics.
• The model learns the mapping between unobserved latent variables (including expert ODE states) and observable clinical measurements via a time-aware encoder.
• A variational inference framework is used to infer the posterior distribution of initial latent states, enabling end-to-end training on irregularly sampled data.
• The method employs a hybrid architecture where the expert ODEs govern part of the dynamics, while neural ODEs learn the remaining dynamics and measurement mappings.
• The framework supports extensions such as incorporating static covariates, informative sampling, and online inference via Bayesian filtering.
• Equation replacement is applied to correct for potential model mis-specification in the expert ODEs by replacing faulty equations with learnable neural networks.

Experimental results

Research questions

• RQ1 Can integrating expert pharmacological ODEs into neural ODEs improve predictive performance in low-data settings for disease progression modeling?
• RQ2 How can unobservable, biologically meaningful variables from expert models be effectively linked to clinically observable measurements in a differentiable, end-to-end framework?
• RQ3 To what extent does the inclusion of domain-specific ODEs enhance sample efficiency compared to pure neural ODEs?
• RQ4 Can the hybrid model provide both high predictive accuracy and clinically interpretable latent variables?

Key findings

• On real-world ICU data from COVID-19 patients, LHM achieved the lowest RMSE across all prediction horizons, with 0.517 (0.72) RMSE at 6 hours, outperforming NODE (0.661) and GRU-ODE (0.549).
• At 72-hour prediction horizon, LHM achieved 0.691 RMSE, significantly lower than the expert model (0.993) and NODE (0.996), demonstrating superior generalization in long-horizon forecasting.
• On synthetic data, LHM showed consistent improvements in sample efficiency, particularly when training data was scarce, confirming the benefit of expert ODEs in data-scarce regimes.
• The model maintained strong performance across all tested time horizons (6, 12, 24, 72 hours), with minimal variance in predictions, indicating robustness.
• LHM’s performance was stable and superior even when the expert model contained potential mis-specifications, especially when combined with equation replacement.