[Paper Review] Deep Neural Network Modeling of Unknown Partial Differential Equations in Nodal Space

Zhen Chen, Victor Churchill+2 more|arXiv (Cornell University)|Jun 7, 2021

Model Reduction and Neural Networks34 references50 citations

TL;DR

This paper proposes a deep neural network (DNN) framework for learning unknown time-dependent partial differential equations (PDEs) directly from solution trajectory data in physical (nodal) space, without requiring modal transforms or geometric information. The method uses a novel DNN architecture with a disassembly block and assembly layer that mimics the time-stepping evolution operator of the PDE, enabling accurate modeling on both structured and unstructured grids, including shock-capturing in nonlinear PDEs like inviscid Burgers' equation.

ABSTRACT

We present a numerical framework for deep neural network (DNN) modeling of unknown time-dependent partial differential equations (PDE) using their trajectory data. Unlike the recent work of [Wu and Xiu, J. Comput. Phys. 2020], where the learning takes place in modal/Fourier space, the current method conducts the learning and modeling in physical space and uses measurement data as nodal values. We present a DNN structure that has a direct correspondence to the evolution operator of the underlying PDE, thus establishing the existence of the DNN model. The DNN model also does not require any geometric information of the data nodes. Consequently, a trained DNN defines a predictive model for the underlying unknown PDE over structureless grids. A set of examples, including linear and nonlinear scalar PDE, system of PDEs, in both one dimension and two dimensions, over structured and unstructured grids, are presented to demonstrate the effectiveness of the proposed DNN modeling. Extension to other equations such as differential-integral equations is also discussed.

Motivation & Objective

• Address the challenge of discovering unknown time-dependent PDEs from limited, noisy trajectory data without prior knowledge of the equation form.
• Overcome limitations of existing methods that rely on modal (Fourier) space or require geometric structure in the data grid.
• Develop a general, meshfree DNN framework capable of modeling diverse PDE types, including scalar and systems of PDEs, in 1D and 2D domains.
• Establish a mathematical foundation for the existence of the DNN model by directly linking its architecture to the evolution operator of the underlying PDE.
• Demonstrate robustness and accuracy of the method on both linear and nonlinear PDEs, including problems with shocks and complex dynamics on unstructured grids.

Proposed method

• Formulate the learning task as approximating the time evolution operator of an unknown PDE using snapshot pairs (u(t), u(t+Δt)) from multiple trajectories.
• Propose a novel DNN architecture composed of a disassembly block and an assembly layer that directly emulates the time-stepping process of the PDE in physical space.
• Use nodal values of the solution as input and output, avoiding transformation into spectral or modal spaces, thus eliminating the need for predefined basis functions.
• Train the DNN using a flow-map-based approach, learning the mapping from state at time t to state at t+Δt without requiring derivative data.
• Enable the use of unstructured grids by designing the DNN to operate on arbitrary nodal configurations without geometric information.
• Apply standard deep learning training procedures (e.g., Adam optimizer, tanh activation, batch training) with hyperparameters tuned per problem.

Experimental results

Research questions

• RQ1 Can a deep neural network accurately model the evolution operator of an unknown PDE using only nodal solution data in physical space?
• RQ2 Does the proposed DNN architecture maintain accuracy and generalization on unstructured grids without geometric or topological priors?
• RQ3 Can the method effectively learn both linear and nonlinear PDEs, including systems of PDEs and equations with discontinuous solutions?
• RQ4 How does the performance of the nodal-space DNN compare to existing methods that operate in modal or spectral space?
• RQ5 To what extent can the DNN generalize to new initial conditions and predict long-time dynamics beyond the training data?

Key findings

• The proposed DNN framework successfully learns and predicts the dynamics of the viscous Burgers' equation on a 1D structured grid, accurately capturing the shock formation at t = 2.
• For the inviscid Burgers' equation, the DNN model correctly predicts the development of a shock discontinuity at t = 2, with small oscillations near the shock consistent with numerical solutions.
• The method achieves accurate long-term prediction for the 1D linear wave system over 10,000 trajectories, with predictions matching the reference solution up to t = 10.
• On a 2D unstructured grid, the DNN model accurately predicts the advection-diffusion equation dynamics up to t = 4, showing good agreement with the reference solution for a Gaussian initial condition.
• The DNN model generalizes well to unseen initial conditions, including piecewise constant and smooth Gaussian profiles, across all test problems.
• The method demonstrates robustness to data sparsity and noise, as evidenced by accurate predictions even when training data are generated from randomized initial conditions and numerical solvers.