[Paper Review] Active Uncertainty Calibration in Bayesian ODE Solvers

Hans Kersting, Philipp Hennig|arXiv (Cornell University)|May 11, 2016

Gaussian Processes and Bayesian Inference18 references19 citations

TL;DR

This paper proposes Bayesian Quadrature Filtering (BQF), a novel probabilistic ODE solver that improves uncertainty calibration in filtering-based methods by actively learning gradient imprecision through Bayesian quadrature. By using deterministic evaluations to refine gradient estimates, BQF achieves better mean accuracy than sampling-based methods and superior uncertainty calibration compared to standard filtering approaches, all with minimal computational overhead.

ABSTRACT

There is resurging interest, in statistics and machine learning, in solvers for ordinary differential equations (ODEs) that return probability measures instead of point estimates. Recently, Conrad et al. introduced a sampling-based class of methods that are 'well-calibrated' in a specific sense. But the computational cost of these methods is significantly above that of classic methods. On the other hand, Schober et al. pointed out a precise connection between classic Runge-Kutta ODE solvers and Gaussian filters, which gives only a rough probabilistic calibration, but at negligible cost overhead. By formulating the solution of ODEs as approximate inference in linear Gaussian SDEs, we investigate a range of probabilistic ODE solvers, that bridge the trade-off between computational cost and probabilistic calibration, and identify the inaccurate gradient measurement as the crucial source of uncertainty. We propose the novel filtering-based method Bayesian Quadrature filtering (BQF) which uses Bayesian quadrature to actively learn the imprecision in the gradient measurement by collecting multiple gradient evaluations.

Motivation & Objective

• To address the trade-off between computational cost and uncertainty calibration in probabilistic ODE solvers.
• To improve the calibration of uncertainty estimates in filtering-based probabilistic ODE solvers, which typically suffer from overconfident or poorly adaptive variance.
• To develop a method that maintains high-order convergence and accurate mean estimates while enabling meaningful uncertainty quantification.
• To bridge the gap between sampling-based solvers (well-calibrated but expensive) and filtering-based solvers (inexpensive but poorly calibrated).
• To identify and actively correct the primary source of uncertainty: imprecise gradient measurements in ODE integration.

Proposed method

• Formulates ODE solution as approximate inference in linear Gaussian stochastic differential equations (SDEs), modeling the solution as a Gaussian process.
• Identifies inaccurate gradient measurement as the dominant source of epistemic uncertainty in ODE solvers.
• Applies Bayesian quadrature (BQ) to actively learn the vector field f by strategically selecting gradient evaluation points to minimize uncertainty in the gradient estimate.
• Integrates BQ into a filtering framework, replacing standard gradient evaluations with BQ-estimated gradients to improve posterior mean and variance calibration.
• Uses a Gaussian process prior over the solution and updates the posterior using BQ-estimated gradients at each step, maintaining low computational overhead.
• Employs a state-space model where the state is the solution u(t), and the observation model is based on BQ-estimated f(t, u(t)) with uncertainty propagation through time.

Experimental results

Research questions

• RQ1 Can active learning of gradient evaluations through Bayesian quadrature improve uncertainty calibration in filtering-based ODE solvers?
• RQ2 Does BQF achieve better mean accuracy than sampling-based solvers while maintaining low computational cost?
• RQ3 How does the uncertainty calibration of BQF compare to that of standard filtering-based and sampling-based ODE solvers?
• RQ4 Can the primary source of uncertainty—imprecise gradient measurement—be effectively modeled and reduced using deterministic sampling strategies?
• RQ5 Is it possible to achieve high-order convergence with well-calibrated uncertainty using only a small number of gradient evaluations?

Key findings

• BQF produces a mean estimate that outperforms state-of-the-art sampling-based solvers on the Van der Pol oscillator, achieving lower error than both ML and MC methods.
• At later time points (e.g., t = 54), BQF with five or more gradient evaluations reduces error significantly below the ML baseline, demonstrating improved convergence.
• The uncertainty estimate of BQF is better calibrated than that of standard filtering-based methods, which tend to produce strictly increasing uncertainty over time.
• Sampling-based methods (MC) show more adaptive uncertainty scaling—increasing in steep regions and decreasing in flat regions—compared to the monotonic uncertainty growth in BQF and ML.
• Despite better uncertainty calibration, MC methods produce higher mean error than BQF due to the accumulation of Gaussian noise in each step, which degrades the mean estimate.
• BQF achieves a favorable trade-off: it maintains high mean accuracy (superior to MC and ML) while producing a more adaptive and well-calibrated uncertainty measure than standard filtering approaches.