[Paper Review] Gaussian Processes simplify differential equations

Jong‐Hyeon Lee, Boumediene Hamzi+2 more|arXiv (Cornell University)|Oct 3, 2024

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper proposes a Gaussian process (GP)-based framework to learn and simplify nonlinear differential equations by discovering mappings between trajectories of nonlinear and linear PDEs or ODEs. Using kernel regression and computational graph completion, it learns the Cole-Hopf transformation for Burgers’ equation and local normal forms for the Brusselator, achieving accurate, data-driven linearization without prior knowledge of the underlying equations, demonstrating that GPs can systematically simplify complex dynamics via implicit transformation learning.

ABSTRACT

In this paper we use Gaussian processes (kernel methods) to learn mappings between trajectories of distinct differential equations. Our goal is to simplify both the representation and the solution of these equations. We begin by examining the Cole-Hopf transformation, a classical result that converts the nonlinear, viscous Burgers' equation into the linear heat equation. We demonstrate that this transformation can be effectively learned using Gaussian process regression, either from single or from multiple initial conditions of the Burgers equation. We then extend our methodology to discover mappings between initial conditions of a nonlinear partial differential equation (PDE) and a linear PDE, where the exact form of the linear PDE remains unknown and is inferred through Computational Graph Completion (CGC), a generalization of Gaussian Process Regression from approximating single input/output functions to approximating multiple input/output functions that interact within a computational graph. Further, we employ CGC to identify a local transformation from the nonlinear ordinary differential equation (ODE) of the Brusselator to its Poincaré normal form, capturing the dynamics around a Hopf bifurcation. We conclude by addressing the broader question of whether systematic transformations between nonlinear and linear PDEs can generally exist, suggesting avenues for future research.

Motivation & Objective

• To develop a data-driven method for discovering coordinate transformations that simplify nonlinear differential equations into linear ones.
• To demonstrate that Gaussian processes can learn known transformations like Cole-Hopf without explicit formulae, using only trajectory data.
• To extend the method to cases where the target linear PDE is unknown, using computational graph completion to infer the transformation.
• To learn local normal forms near bifurcations, such as the Hopf bifurcation in the Brusselator, via trajectory-based GP regression.
• To explore the general feasibility of systematic linearizing transformations between nonlinear and linear PDEs using kernel methods.

Proposed method

• Gaussian process regression is used to learn mappings between trajectories of nonlinear and linear differential equations, treating the transformation as a function approximation problem.
• The method leverages kernel methods and reproducing kernel Hilbert spaces (RKHS) to ensure smooth, probabilistic function estimation with uncertainty quantification.
• Computational Graph Completion (CGC) generalizes GP regression to multiple interacting input-output functions, enabling inference of unknown linear PDEs from data.
• For the Cole-Hopf transformation, GP regression is applied to pairs of initial conditions from Burgers’ and heat equations, learning the transformation from trajectory data.
• For the Brusselator, GP regression learns a local transformation to its Poincaré normal form near a Hopf bifurcation, using only trajectory data and no explicit equations.
• An optimization-based approach is used to discover the linear PDE structure when it is not known a priori, by fitting the GP model to observed trajectories.

(a)

Experimental results

Research questions

• RQ1 Can Gaussian processes learn the Cole-Hopf transformation between Burgers’ equation and the heat equation from trajectory data alone?
• RQ2 Can the method discover unknown linear PDEs that linearize a given nonlinear PDE through data-driven inference?
• RQ3 Can GP-based regression learn local normal forms for nonlinear ODEs near bifurcation points without knowledge of the underlying equations?
• RQ4 How does kernel choice and hyperparameter learning affect the accuracy of the discovered transformations?
• RQ5 What is the potential for generalizing this framework to discover systematic linearizing transformations across broader classes of nonlinear PDEs?

Key findings

• The Cole-Hopf transformation between Burgers’ equation and the heat equation can be accurately learned via Gaussian process regression using only trajectory data from a single or multiple initial conditions.
• The method achieves high accuracy in learning the transformation even when the exact form of the linear PDE is unknown, by combining GP regression with computational graph completion.
• For the Brusselator, the GP-based approach successfully learns the local transformation to its Poincaré normal form near a Hopf bifurcation, capturing the dynamics without requiring explicit equations.
• The choice of kernel significantly impacts learning accuracy, and an effective method for kernel learning is proposed to improve performance.
• The framework demonstrates that Gaussian processes can serve as a powerful, principled tool for discovering and simplifying complex dynamical systems through data-driven transformation learning.
• The results suggest that systematic linearizing transformations between nonlinear and linear PDEs may be systematically discovered using kernel-based methods, opening new avenues for dynamical systems analysis.

(b)