[Paper Review] A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations

Ramakrishna Tipireddy, Paris Perdikaris+2 more|arXiv (Cornell University)|Apr 2, 2019

Model Reduction and Neural Networks14 references34 citations

TL;DR

The paper compares discrete multistep and continuous physics-informed neural network (PINN) approaches to learn unknown dynamics and constitutive relations, using a fedbatch bioreactor as the test case, and finds higher accuracy when learning constitutive relations than full dynamics; discrete methods excel broadly, while continuous methods are better for constitutive relations.

ABSTRACT

We investigate the use of discrete and continuous versions of physics-informed neural network methods for learning unknown dynamics or constitutive relations of a dynamical system. For the case of unknown dynamics, we represent all the dynamics with a deep neural network (DNN). When the dynamics of the system are known up to the specification of constitutive relations (that can depend on the state of the system), we represent these constitutive relations with a DNN. The discrete versions combine classical multistep discretization methods for dynamical systems with neural network based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of the whole dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information is available into an algorithm can enhance its efficiency and/or accuracy.

Motivation & Objective

• Motivate the integration of physics knowledge with data to learn unknown dynamics and constitutive relations in dynamical systems.
• Systematically compare four PINN variants (discrete multistep and continuous PINN) for learning dynamics and constitutive relations.
• Evaluate the approaches on a tractable yet sensitive fedbatch bioreactor model to reveal applicability and limitations.
• Demonstrate that incorporating known structure improves computational efficiency and accuracy.

Proposed method

• Model unknown dynamics f(y,t;λ) with a neural network when f is completely unknown.
• Use a multistep discretization combined with PINNs to learn f or to learn constitutive relations λ(y).
• Represent constitutive relations with a neural network when the governing dynamics are known but λ is unknown.
• Apply continuous PINN formulations to learn y(t) and f(y,t) by minimizing residuals and data-fit terms.
• Compare discrete multistep NN and continuous PINN approaches for learning dynamics or constitutive relations.
• Use the fedbatch bioreactor (FBR) model as the test case with Haldane kinetics as ground truth.

Experimental results

Research questions

• RQ1 Which PINN variant (discrete multistep vs. continuous) is more accurate for learning unknown dynamics versus constitutive relations?
• RQ2 Does incorporating known physics (dynamics or constitutive relations) into the neural network learning improve prediction accuracy and data efficiency?
• RQ3 How do initial conditions and training data duration affect interpolation vs. extrapolation performance for each method?
• RQ4 Can CPINN reliably learn constitutive relations that depend nonlinearly on state variables in a complex bioreactor model?

Key findings

• Discrete multistep NN delivers accurate learning for both unknown dynamics and constitutive relations.
• Continuous PINN excels at learning constitutive relations but may underperform for unknown dynamics in this setting.
• Learning a constitutive relation often yields higher accuracy than learning the full dynamics due to leveraging known physics structure.
• CPINN can accurately infer constitutive relations like μ(S) and use them to predict system behavior under new conditions.
• CPINN struggles to learn unknown dynamics from limited or non-uniform data and may require additional constraints or data.
• Using multiple short trajectories improves learning when data is scarce or initial conditions vary.