[Paper Review] Exploring Physics-Informed Neural Networks: From Fundamentals to Applications in Complex Systems

Sai Ganga, Ziya Uddin|arXiv (Cornell University)|Oct 1, 2024

Neural Networks and Applications4 citations

TL;DR

This paper provides a comprehensive review of Physics-Informed Neural Networks (PINNs), detailing their fundamentals, optimization strategies, and applications in solving complex partial differential equations—particularly in computational fluid dynamics. It demonstrates that PINNs offer a mesh-free, data-free alternative to traditional numerical methods, enabling accurate solutions for nonlinear, high-dimensional, and irregular-domain problems with minimal manual intervention.

ABSTRACT

Physics-informed neural networks (PINNs) have emerged as a versatile and widely applicable concept across various science and engineering domains over the past decade. This article offers a comprehensive overview of the fundamentals of PINNs, tracing their evolution, modifications, and various variants. It explores the impact of different parameters on PINNs and the optimization algorithms involved. The review also delves into the theoretical advancements related to the convergence, consistency, and stability of numerical solutions using PINNs, while highlighting the current state of the art. Given their ability to address equations involving complex physics, the article discusses various applications of PINNs, with a particular focus on their utility in computational fluid dynamics problems. Additionally, it identifies current gaps in the research and outlines future directions for the continued development of PINNs.

Motivation & Objective

• To provide a thorough review of the theoretical foundations, evolution, and variants of Physics-Informed Neural Networks (PINNs).
• To analyze the impact of hyperparameters, optimization algorithms, and numerical stability on PINN performance.
• To evaluate PINNs' effectiveness in solving complex partial differential equations, especially in fluid mechanics and high-dimensional problems.
• To identify current research gaps and propose future directions for improving PINN robustness, efficiency, and theoretical grounding.
• To demonstrate PINN's utility in real-world fluid dynamics applications, including non-Newtonian and shallow water flows, without requiring extensive experimental data.

Proposed method

• PINNs approximate solutions to partial differential equations (PDEs) using deep neural networks, with the loss function composed of residual terms from the PDE, initial conditions, and boundary conditions.
• The method integrates physical laws directly into the neural network loss function, enabling a data-free training paradigm that does not require labeled datasets.
• Random sampling of collocation points within the domain is used to evaluate PDE residuals during training, avoiding the need for mesh generation.
• The framework supports both soft and hard boundary condition enforcement, with hard constraints shown to improve solution accuracy.
• Advanced architectures such as wavelet activation functions and separate networks for different solution components are explored to enhance performance on coupled nonlinear systems.
• Techniques like characteristic-based split methods are applied to reduce gradient computation complexity and accelerate training in fluid dynamics problems.

Figure 1: Schematic representation of PINN

Experimental results

Research questions

• RQ1 How do different optimization algorithms and hyperparameters affect the convergence and stability of PINN solutions?
• RQ2 To what extent can PINNs accurately solve high-dimensional and complex fluid dynamics problems without experimental or simulation data?
• RQ3 What are the limitations of PINNs in terms of training time, vanishing gradients, and local minima, and how can they be mitigated?
• RQ4 How do architectural choices—such as activation functions, network depth, and boundary condition enforcement—affect PINN performance?
• RQ5 What future developments are needed to improve the theoretical foundation, generalization, and computational efficiency of PINNs?

Key findings

• PINNs provide a mesh-free alternative to traditional numerical methods, significantly reducing the computational burden associated with mesh generation in complex or irregular domains.
• The method achieves accurate solutions for nonlinear, coupled PDEs such as the Navier-Stokes and shallow water equations, even with minimal or no data, demonstrating strong generalization in fluid flow simulations.
• Using wavelet activation functions and separate neural networks for different solution components improves the accuracy and stability of PINN solutions in non-Newtonian Maxwell fluid flows.
• The characteristic-based split method reduces computational overhead by excluding certain partial derivatives from backpropagation, accelerating training while maintaining solution fidelity.
• Despite their advantages, PINNs face challenges in training time, hyperparameter sensitivity, and convergence due to non-convex loss landscapes and gradient vanishing, especially in deep networks.
• Future improvements are needed in automated hyperparameter tuning, pre-trained models, transfer learning, and alternative differentiation methods to enhance scalability and efficiency.