[Paper Review] Improving physics-informed DeepONets with hard constraints

Rüdiger Brecht, Dmytro R. Popovych+2 more|arXiv (Cornell University)|Sep 14, 2023

Model Reduction and Neural NetworksPhysics and Astronomy3 citations

TL;DR

This paper proposes integrating hard constraints into physics-informed DeepONets to exactly represent initial and boundary conditions, eliminating the need for loss-based learning of these conditions. By embedding initial data directly into the network architecture via a residual-style formulation, the method ensures solution continuity during time-stepping and reduces errors by up to 80% compared to soft-constrained baselines on benchmark PDEs including the KdV equation.

ABSTRACT

Current physics-informed (standard or deep operator) neural networks still rely on accurately learning the initial and/or boundary conditions of the system of differential equations they are solving. In contrast, standard numerical methods involve such conditions in computations without needing to learn them. In this study, we propose to improve current physics-informed deep learning strategies such that initial and/or boundary conditions do not need to be learned and are represented exactly in the predicted solution. Moreover, this method guarantees that when a deep operator network is applied multiple times to time-step a solution of an initial value problem, the resulting function is at least continuous.

Motivation & Objective

• To address the computational inefficiency of physics-informed neural networks that require retraining for each new initial or boundary condition.
• To overcome the instability and error accumulation in soft-constrained DeepONets during iterative time-stepping.
• To develop a DeepONet architecture that exactly satisfies initial and boundary conditions without relying on loss function penalties.
• To demonstrate that hard-constrained DeepONets outperform soft-constrained counterparts in accuracy and solution continuity.
• To enable efficient, reusable solution operators for time-dependent PDEs through exact initial condition representation.

Proposed method

• The hard constraint is implemented via a residual architecture: $ u(t,x) \approx \mathcal{G}^{\theta}(\mathbf{\Phi})(t,x) = u_0(x,a,c) + \frac{t}{T} \mathcal{G}^{\theta}_{\rm t}(\mathbf{\Phi})(t,x) $, where $ \mathbf{\Phi} $ encodes initial conditions at sensor points.
• Initial conditions are sampled at $ N_{\rm s} = 100 $ evenly spaced points in the spatial domain, forming the input to the DeepONet.
• The loss function is simplified to depend only on the PDE residual, since initial and boundary conditions are satisfied a priori by the network structure.
• Time-stepping is performed by recursively applying the trained DeepONet, using the previous time step’s output as the new initial condition.
• The method is evaluated on the damped pendulum, Poisson equation, and Korteweg–de Vries (KdV) equation with exact soliton solutions.
• A comparison is made between soft-constrained and hard-constrained DeepONets using RMSE against reference solutions over $[0,10]$ and $[0,100]$ time intervals.

Experimental results

Research questions

• RQ1 Can hard constraints on initial conditions improve the accuracy of physics-informed DeepONets compared to soft constraints?
• RQ2 Does enforcing initial conditions exactly prevent error accumulation during iterative time-stepping?
• RQ3 How does the hard-constrained DeepONet perform on time-dependent PDEs like the KdV equation?
• RQ4 Can the hard-constrained architecture simplify the optimization problem by removing the need to learn boundary conditions?
• RQ5 What is the impact of hard constraints on solution continuity when performing multiple time steps?

Key findings

• The hard-constrained DeepONet reduced RMSE by 53% after one time step and 80% after ten time steps on the KdV equation compared to the soft-constrained baseline.
• After ten time steps, the soft-constrained DeepONet’s error was approximately five times higher than the hard-constrained version.
• The hard-constrained method guaranteed continuous solutions during time-stepping, whereas the soft-constrained approach exhibited jumps between steps.
• The loss function was simplified to depend only on the PDE residual, as initial and boundary conditions were satisfied by architecture, not optimization.
• The method achieved exact representation of initial conditions without requiring them to be learned during training.
• The approach is particularly effective for time-dependent PDEs where iterative time-stepping is used, as it avoids error propagation from inconsistent initial data.