[Paper Review] Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Maziar Raissi|arXiv (Cornell University)|Jan 20, 2018

Model Reduction and Neural Networks28 references438 citations

TL;DR

This paper introduces Deep Hidden Physics Models that learn both the solution u and the nonlinear dynamics N in PDEs from scattered, possibly noisy data by using two neural networks, and demonstrates accurate discovery and forecasting on several canonical PDEs via PINNs.

ABSTRACT

A long-standing problem at the interface of artificial intelligence and applied mathematics is to devise an algorithm capable of achieving human level or even superhuman proficiency in transforming observed data into predictive mathematical models of the physical world. In the current era of abundance of data and advanced machine learning capabilities, the natural question arises: How can we automatically uncover the underlying laws of physics from high-dimensional data generated from experiments? In this work, we put forth a deep learning approach for discovering nonlinear partial differential equations from scattered and potentially noisy observations in space and time. Specifically, we approximate the unknown solution as well as the nonlinear dynamics by two deep neural networks. The first network acts as a prior on the unknown solution and essentially enables us to avoid numerical differentiations which are inherently ill-conditioned and unstable. The second network represents the nonlinear dynamics and helps us distill the mechanisms that govern the evolution of a given spatiotemporal data-set. We test the effectiveness of our approach for several benchmark problems spanning a number of scientific domains and demonstrate how the proposed framework can help us accurately learn the underlying dynamics and forecast future states of the system. In particular, we study the Burgers', Korteweg-de Vries (KdV), Kuramoto-Sivashinsky, nonlinear Schrödinger, and Navier-Stokes equations.

Motivation & Objective

• Motivate data-driven discovery of governing PDEs when first-principles derivations are intractable.
• Propose a two-network framework to learn the solution and the nonlinear dynamics without explicit derivative computations.
• Avoid numerical differentiation by using neural network priors and automatic differentiation for stable training.
• Demonstrate the approach on benchmark PDEs spanning fluid mechanics and nonlinear wave equations and assess robustness to noise.

Proposed method

• Represent the solution u and the nonlinear function N with two deep neural networks.
• Define the Deep Hidden Physics Model f = u_t - N(t, x, u, u_x, u_xx, ...).
• Compute derivatives via automatic differentiation (e.g., with TensorFlow) rather than finite differences.
• Train by minimizing the loss that combines data fit (u) and PDE residual (f) at collocation points.
• Solve the learned PDEs using physics-informed neural networks (PINNs) as solvers to forecast future states.

Experimental results

Research questions

• RQ1 Can a neural network-based representation of the solution and a separate network for the dynamics identify the underlying nonlinear PDE from scattered data?
• RQ2 How does the order of derivatives included in N affect identifiability, accuracy, and boundary requirements?
• RQ3 What is the robustness of the approach to noise and to extrapolation beyond the training domain?
• RQ4 How well does the method generalize to different PDEs (Burgers, KdV, Kuramoto–Sivashinsky, nonlinear Schrödinger, Navier–Stokes)?

Key findings

• The Burgers’ equation is learned with a relative L2 error of 4.78e-03 on training data and 3.89e-03 on the training portion.
• For Burgers with a Gaussian initial condition, learned PDE extrapolates to a relative L2 error of 7.33e-02 on a different dataset.
• For the KdV equation, the learned PDE achieves a relative L2 error of 6.28e-02 on the test data and 3.78e-02 on the training portion.
• For the nonlinear Schrödinger equation, the method yields a relative L2 error of 6.28e-03 in reproducing the absolute value of the solution.
• For the Kuramoto–Sivashinsky equation, the approach attains a relative L2 error of 7.63e-02 within the studied regime.
• The Navier–Stokes example demonstrates the framework’s applicability to 2D flows and supports its generalizability across multiple canonical PDEs.