[Paper Review] Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations

Zixue Xiang, Wei Peng+3 more|arXiv (Cornell University)|Apr 13, 2021

Model Reduction and Neural Networks52 references26 citations

TL;DR

The paper introduces lbPINNs, a self-adaptive, loss-balanced physics-informed neural network framework for solving incompressible Navier-Stokes equations, improving accuracy by adapting loss term weights during training. It demonstrates superior results on 2D Kovasznay flow, 2D cylinder wake, and 3D Beltrami flow compared with baseline PINNs.

ABSTRACT

There have been several efforts to Physics-informed neural networks (PINNs) in the solution of the incompressible Navier-Stokes fluid. The loss function in PINNs is a weighted sum of multiple terms, including the mismatch in the observed velocity and pressure data, the boundary and initial constraints, as well as the residuals of the Navier-Stokes equations. In this paper, we observe that the weighted combination of competitive multiple loss functions plays a significant role in training PINNs effectively. We establish Gaussian probabilistic models to define the loss terms, where the noise collection describes the weight parameter for each loss term. We propose a self-adaptive loss function method, which automatically assigns the weights of losses by updating the noise parameters in each epoch based on the maximum likelihood estimation. Subsequently, we employ the self-adaptive loss balanced Physics-informed neural networks (lbPINNs) to solve the incompressible Navier-Stokes equations,\hspace{-1pt} including\hspace{-1pt} two-dimensional\hspace{-1pt} steady Kovasznay flow, two-dimensional unsteady cylinder wake, and three-dimensional unsteady Beltrami flow. Our results suggest that the accuracy of PINNs for effectively simulating complex incompressible flows is improved by adaptively appropriate weights in the loss terms. The outstanding adaptability of lbPINNs is not irrelevant to the initialization choice of noise parameters, which illustrates the robustness. The proposed method can also be employed in other problems where PINNs apply besides fluid problems.

Motivation & Objective

• Motivate the need for improved training balance among multiple physics-based loss terms in PINNs for incompressible Navier-Stokes problems.

Proposed method

• Formulates NS equations in a PINN framework with residuals f1 and f2 and multiple loss terms (PDE, BC, IC, data).
• Introduces Gaussian probabilistic modeling of loss terms with per-term observation noise parameters to balance losses via maximum likelihood.

Experimental results

Research questions

• RQ1 Can dynamic, data-driven weighting of loss terms improve PINN accuracy for incompressible NS flows?
• RQ2 How does adaptive loss balancing affect convergence and robustness across 2D and 3D flow problems?
• RQ3 What is the impact of initial noise configurations on lbPINNs performance and robustness?
• RQ4 Are lbPINNs able to achieve lower errors than baseline PINNs on representative laminar and unsteady NS cases?

Key findings

• lbPINNs achieve substantially lower relative errors in velocity and pressure than baseline PINNs in tested cases (e.g., down to 1e-4 to 1e-5 in some setups).
• Adaptive noise parameters evolve during training to balance PDE and boundary/data losses, accelerating convergence.
• lbPINNs show robustness to different initial noise settings and maintain improved accuracy over baseline PINNs across tested flows.
• For 2D Kovasznay flow, lbPINNs reach an L2 error near 6.4e-4 compared with 4.4e-3 for baseline PINNs under similar conditions.
• lbPINNs demonstrate superior velocity and pressure accuracy in 2D cylinder wake and 3D Beltrami flow demonstrations, with PDE and data losses converging alongside adaptive noise.
• The method provides a principled approach to loss weighting, extending beyond fluid dynamics to other PINN applications.