[Paper Review] Reduced-PINN: An Integration-Based Physics-Informed Neural Networks for Stiff ODEs

Pouyan Nasiri, Roozbeh Dargazany|arXiv (Cornell University)|Aug 23, 2022

Model Reduction and Neural Networks6 citations

TL;DR

This paper proposes Reduced-PINN, a physics-informed neural network framework that reformulates stiff ODEs into weak-form integral equations to overcome the training instability and poor performance of standard PINNs in stiff systems. By replacing multiple loss terms with a single integral-based loss, Reduced-PINN enables stable, accurate, and computationally efficient simulation of stiff chemical kinetics, including the challenging ROBER problem, with results matching high-order numerical solvers like BDF.

ABSTRACT

Physics-informed neural networks (PINNs) have recently received much attention due to their capabilities in solving both forward and inverse problems. For training a deep neural network associated with a PINN, one typically constructs a total loss function using a weighted sum of different loss terms and then tries to minimize that. This approach often becomes problematic for solving stiff equations since it cannot consider adaptive increments. Many studies reported the poor performance of the PINN and its challenges in simulating stiff chemical active issues with administering conditions of stiff ordinary differential conditions (ODEs). Studies show that stiffness is the primary cause of the failure of the PINN in simulating stiff kinetic systems. Here, we address this issue by proposing a reduced weak-form of the loss function, which led to a new PINN architecture, further named as Reduced-PINN, that utilizes a reduced-order integration method to enable the PINN to solve stiff chemical kinetics. The proposed Reduced-PINN can be applied to various reaction-diffusion systems involving stiff dynamics. To this end, we transform initial value problems (IVPs) to their equivalent integral forms and solve the resulting integral equations using physics-informed neural networks. In our derived integral-based optimization process, there is only one term without explicitly incorporating loss terms associated with ordinary differential equation (ODE) and initial conditions (ICs). To illustrate the capabilities of Reduced-PINN, we used it to simulate multiple stiff/mild second-order ODEs. We show that Reduced-PINN captures the solution accurately for a stiff scalar ODE. We also validated the Reduced-PINN against a stiff system of linear ODEs.

Motivation & Objective

• Address the failure of standard PINNs in simulating stiff ordinary differential equations (ODEs), particularly in chemical kinetics with widely varying time scales.
• Overcome the limitations of weighted-sum loss functions in PINNs, which struggle with stiffness due to conflicting gradients from fast and slow dynamics.
• Develop a unified, single-loss formulation by transforming initial value problems into weak-form integral equations, reducing complexity and improving training stability.
• Enable long-term simulations of stiff systems through sequential application of Reduced-PINN over subdivided time intervals.
• Validate the framework on benchmark stiff problems, including the nonlinear ROBER system, demonstrating superior accuracy and efficiency over standard PINNs.

Proposed method

• Transform initial value problems (IVPs) into equivalent integral equations using weak-form formulations, eliminating explicit ODE and initial condition loss terms.
• Construct a single, unified loss function based on the residual of the integral equation, replacing multiple loss components in standard PINNs.
• Apply a reduced-order integration method to approximate the integral terms efficiently, reducing computational cost.
• Use sequential Reduced-PINNs over subdivided time intervals to enable long-time simulations with improved training speed and stability.
• Train deep neural networks to approximate the solution trajectories, with the loss function enforced via automatic differentiation.
• Utilize backward differentiation formula (BDF) as a benchmark for comparison in high-accuracy simulations.

Experimental results

Research questions

• RQ1 Can a single, integral-based loss function replace multiple loss terms in PINNs to improve training stability for stiff ODEs?
• RQ2 Does transforming ODEs into weak-form integral equations enable more accurate and efficient simulation of stiff chemical kinetics than standard PINNs?
• RQ3 Can Reduced-PINN successfully simulate long-time dynamics of stiff systems by dividing the time domain into smaller intervals?
• RQ4 How does Reduced-PINN perform on benchmark stiff problems such as the ROBER system with 9-order time-scale disparity?
• RQ5 Can Reduced-PINN achieve accuracy comparable to high-order numerical solvers like BDF while reducing computational cost?

Key findings

• Reduced-PINN accurately captures the solution of a stiff scalar ODE, demonstrating superior performance over standard PINNs.
• For a stiff system of linear ODEs, Reduced-PINN achieves excellent agreement with the exact solution, validating its robustness.
• In the ROBER problem—a benchmark nonlinear stiff system with time-scale disparities of 9 orders—Reduced-PINN matches BDF solver results with high fidelity.
• The method achieves accurate simulation with significantly reduced computational cost compared to standard PINNs, which fail to converge on such problems.
• Sequential application of Reduced-PINNs over subdivided time intervals enables stable, scalable long-term simulations.
• The single-integral loss formulation eliminates gradient imbalance issues common in standard PINNs, leading to more reliable training.