[Paper Review] Data-driven discovery of free-form governing differential equations

Steven Atkinson, Waad Subber+4 more|arXiv (Cornell University)|Sep 27, 2019

Model Reduction and Neural Networks25 references32 citations

TL;DR

A method to discover governing differential equations from data without pre-specifying terms, using differentiable data models and a genetic-programming graph of operators, with Bayesian calibration and active learning demonstrations on synthetic and ultrasound data.

ABSTRACT

We present a method of discovering governing differential equations from data without the need to specify a priori the terms to appear in the equation. The input to our method is a dataset (or ensemble of datasets) corresponding to a particular solution (or ensemble of particular solutions) of a differential equation. The output is a human-readable differential equation with parameters calibrated to the individual particular solutions provided. The key to our method is to learn differentiable models of the data that subsequently serve as inputs to a genetic programming algorithm in which graphs specify computation over arbitrary compositions of functions, parameters, and (potentially differential) operators on functions. Differential operators are composed and evaluated using recursive application of automatic differentiation, allowing our algorithm to explore arbitrary compositions of operators without the need for human intervention. We also demonstrate an active learning process to identify and remedy deficiencies in the proposed governing equations.

Motivation & Objective

• Motivate automatic discovery of governing laws from abundant data without assuming term templates.
• Develop a differentiable, operator-graph-based search that composes algebraic and differential operators to form equations.
• Calibrate equation parameters via Bayesian inference.
• Demonstrate robustness on synthetic differential equations and a real-world ultrasound dataset.
• Provide a pathway to integrate discovered laws with existing physics-informed and analytical workflows.

Proposed method

• Fit differentiable models (neural nets or Gaussian processes) to each data component to obtain smooth function representations.
• Construct differential-equation candidates as graphs where leaves are fitted models and internal nodes are operators drawn from a library of algebraic and differential operators.
• Compute operators via automatic differentiation to evaluate residuals r(x) for any x in the domain.
• Explore the hypothesis space with a genetic programming algorithm to generate and rank equation graphs using evidence lower bound (ELBO) as fitness.
• Calibrate parameters theta on leaves representing constants with black-box variational inference (Bayesian).
• Optionally enable active learning by selecting new data to maximize the fit improvement measure a(x) and iteratively refine the discovered equation.

Experimental results

Research questions

• RQ1 Can a free-form differential equation be discovered from data without specifying candidate terms a priori?
• RQ2 How well can a graph-based composition of operators and differentiable models recover known differential equations from synthetic benchmarks?
• RQ3 Can the approach calibrate unknown parameters in the discovered equations to data from individual experiments?
• RQ4 Is active learning effective in improving the discovered governing law with additional data?
• RQ5 How does the method perform on real-world data (e.g., ultrasound) compared to purely synthetic benchmarks?

Key findings

• Successful discovery of several benchmark differential equations from synthetic data, including ODEs and PDEs, using a fully data-driven operator-graph approach.
• The method can recover correct governing structures and calibrate coefficients for individual data instances.
• Gaussian process models provided more reliable gradient information than neural nets for this task, aiding accurate operator assembly.
• Adaptive sampling and an ELBO-based fitness guide the evolutionary search toward correct equations with fewer samples.
• Applied to ultrasound data, the approach yielded plausible corrective terms to a wave equation, with quantified parameter posteriors.
• The framework yields human-readable, compositional differential equations compatible with traditional modeling and physics-informed methods.