[Paper Review] Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing

Salah A Faroughi, Nikhil Pawar+5 more|arXiv (Cornell University)|Nov 14, 2022

Model Reduction and Neural Networks47 citations

TL;DR

This paper reviews four neural-network frameworks—PgNNs, PiNNs, PeNNs, and neural operators (NOs)—for enforcing physics in scientific computing, discusses architectures, applications, limitations, and future opportunities in fluid and solid mechanics.

ABSTRACT

Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, including fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints during training. Generally speaking, there are three distinct neural network frameworks to enforce the underlying physics: (i) physics-guided neural networks (PgNNs), (ii) physics-informed neural networks (PiNNs), and (iii) physics-encoded neural networks (PeNNs). These methods provide distinct advantages for accelerating the numerical modeling of complex multiscale multi-physics phenomena. In addition, the recent developments in neural operators (NOs) add another dimension to these new simulation paradigms, especially when the real-time prediction of complex multi-physics systems is required. All these models also come with their own unique drawbacks and limitations that call for further fundamental research. This study aims to present a review of the four neural network frameworks (i.e., PgNNs, PiNNs, PeNNs, and NOs) used in scientific computing research. The state-of-the-art architectures and their applications are reviewed, limitations are discussed, and future research opportunities in terms of improving algorithms, considering causalities, expanding applications, and coupling scientific and deep learning solvers are presented. This critical review provides researchers and engineers with a solid starting point to comprehend how to integrate different layers of physics into neural networks.

Motivation & Objective

• Summarize how physics is incorporated into neural networks to address data sparsity in scientific computing.
• Compare PgNNs, PiNNs, PeNNs, and neural operators in terms of architectures, strengths, and limitations.
• Analyze applications of these frameworks in fluid mechanics and solid mechanics.
• Highlight current limitations and outline future research directions for integrating physics with deep learning.

Proposed method

• Describe PgNNs as data-driven surrogates that incorporate known physics through supervised learning on physics-based datasets.
• Explain PiNNs as models that embed physical laws via residuals of governing equations in the loss function using automatic differentiation.
• Introduce PeNNs as architectures that encode physics directly into the network structure for enhanced generalization under data sparsity.
• Discuss neural operators (NOs) as models learning continuous operators, enabling real-time predictions and potential coupling with PiNN/PeNN frameworks.

Experimental results

Research questions

• RQ1 What are the distinct neural network frameworks for enforcing physics in scientific computing (PgNNs, PiNNs, PeNNs, NOs) and how do they differ in theory and practice?
• RQ2 What are the main applications, advantages, and limitations of these physics-informed models in fluid and solid mechanics?
• RQ3 How do these approaches address data sparsity and generalization, and what are the prospects for real-time or multi-physics simulations?
• RQ4 What future research opportunities exist to improve algorithms, causality considerations, and solver coupling?

Key findings

• PgNNs can accelerate pre-processing, modeling, and post-processing steps in scientific computing by leveraging physics-based data.
• PiNNs enforce physical laws through equation residuals, enabling learning with sparse data but facing issues like convergence, stability, and boundary-condition handling.
• PeNNs encode physics into network architecture, improving performance under data scarcity and generalization compared to PgNNs and PiNNs.
• Neural operators (NOs) learn continuous operators and offer robustness for real-time inference, and can be combined with PiNNs and PeNNs for complex non-linear multi-physics learning.
• Hybrid approaches that couple neural architectures with traditional solvers can achieve substantial speed-ups (e.g., coarser grids with comparable accuracy) and improved efficiency in CFD and multi-physics problems.
• The review highlights the state-of-the-art architectures, cross-domain applications, and the need for further fundamental research on causality, convergence, and broader applications.