[Paper Review] Least-Squares ReLU Neural Network (LSNN) Method For Scalar Nonlinear Hyperbolic Conservation Law

Zhiqiang Cai, Jingshuang Chen+1 more|arXiv (Cornell University)|May 25, 2021

Model Reduction and Neural Networks4 citations

TL;DR

This paper introduces the block space-time least-squares ReLU neural network (LSNN) method for solving scalar nonlinear hyperbolic conservation laws, leveraging a least-squares formulation over ReLU neural network functions with numerical integration and conservative finite volume schemes. The method accurately captures discontinuous shocks without Gibbs phenomena and achieves lower degrees of freedom than traditional mesh-based methods.

ABSTRACT

We introduced the least-squares ReLU neural network (LSNN) method for solving the linear advection-reaction problem with discontinuous solution and showed that the method outperforms mesh-based numerical methods in terms of the number of degrees of freedom. This paper studies the LSNN method for scalar nonlinear hyperbolic conservation law. The method is a discretization of an equivalent least-squares (LS) formulation in the set of neural network functions with the ReLU activation function. Evaluation of the LS functional is done by using numerical integration and conservative finite volume scheme. Numerical results of some test problems show that the method is capable of approximating the discontinuous interface of the underlying problem automatically through the free breaking lines of the ReLU neural network. Moreover, the method does not exhibit the common Gibbs phenomena along the discontinuous interface.

Motivation & Objective

• Address the challenge of solving scalar nonlinear hyperbolic conservation laws with discontinuous solutions, where traditional mesh-based methods suffer from Gibbs phenomena and require complex stabilization.
• Develop a mesh-free numerical method using ReLU neural networks that can automatically adapt to discontinuous interfaces through the network's piecewise linear structure.
• Overcome training instability in nonlinear hyperbolic problems by introducing a block space-time LSNN approach that processes the domain sequentially based on inflow and initial conditions.
• Ensure weak enforcement of the Rankine-Hugoniot jump condition through implicit finite difference operators derived from conservative schemes like Roe and ENO.
• Demonstrate that the LSNN method achieves high accuracy with fewer degrees of freedom than conventional finite volume methods, especially in shock resolution.

Proposed method

• Formulate the scalar nonlinear hyperbolic conservation law as a least-squares minimization problem over the space of ReLU neural network functions, minimizing the residual of the PDE, initial condition, and inflow boundary condition.
• Discretize the least-squares functional using numerical integration and conservative finite volume schemes to approximate spatial derivatives and fluxes.
• Introduce implicit discrete finite difference operators to weakly enforce the Rankine-Hugoniot jump condition at discontinuities, ensuring physical consistency.
• Propose a block space-time LSNN method that partitions the space-time domain into sequential blocks based on inflow and initial data, enabling stable training by propagating information step-by-step.
• Use the trained parameters from one block as initialization for the next, improving convergence and stability in nonlinear, time-dependent problems.
• Employ second-order ENO and Roe schemes for numerical flux evaluation to enhance accuracy near shocks while maintaining conservation.

Experimental results

Research questions

• RQ1 Can the least-squares ReLU neural network (LSNN) method effectively resolve discontinuous shocks in scalar nonlinear hyperbolic conservation laws without spurious oscillations?
• RQ2 How does the block space-time LSNN method improve training stability and accuracy compared to a global LSNN approach in nonlinear hyperbolic problems?
• RQ3 To what extent does the LSNN method reduce the number of degrees of freedom compared to traditional mesh-based methods like finite volume schemes?
• RQ4 How do different numerical flux schemes (Roe vs. ENO) affect the accuracy and shock resolution of the LSNN method?
• RQ5 Can the LSNN method capture complex wave structures such as rarefaction waves and non-convex fluxes with sufficient accuracy?

Key findings

• The LSNN method successfully captures discontinuous shocks in the inviscid Burgers’ equation without exhibiting Gibbs phenomena, even with piecewise linear initial conditions.
• Relative $L^2$ errors for the ENO-flux LSNN method on six blocks were below 0.053 at $t=0.6$, with the lowest error (0.008958) in the first block, indicating good shock resolution.
• The ENO-flux variant outperformed the Roe-flux variant in both relative $L^2$ error and shock profile accuracy, particularly near discontinuities, despite higher computational cost.
• The block space-time LSNN method reduced the number of degrees of freedom significantly compared to standard finite volume methods, while maintaining high accuracy in shock resolution.
• The method demonstrated robustness in handling problems with sharp gradients and discontinuities by leveraging the free-breaking-line structure of ReLU networks to adaptively locate shocks.
• Despite promising results, the LSNN method showed limitations in accuracy for complex initial conditions and non-convex fluxes, suggesting a need for improved numerical fluxes and optimization strategies.