[Paper Review] Numerical Solution and Bifurcation Analysis of Nonlinear Partial Differential Equations with Extreme Learning Machines

Gianluca Fabiani, Francesco Calabrò+2 more|arXiv (Cornell University)|Apr 13, 2021

Model Reduction and Neural NetworksPhysics and Astronomy69 references79 citations

TL;DR

This paper proposes a novel numerical scheme using Extreme Learning Machines (ELMs) with sigmoidal and radial-basis functions to compute steady-state solutions and bifurcation diagrams of nonlinear PDEs. By leveraging collocation and a one-step least-squares solution for output weights, the method bypasses iterative training, achieving higher accuracy than finite differences and finite elements for medium to large grids, especially near steep gradients and turning points in Burgers and Bratu–Gelfand equations.

ABSTRACT

We address a new numerical scheme based on a class of machine learning methods, the so-called Extreme Learning Machines with both sigmoidal and radial-basis functions, for the computation of steady-state solutions and the construction of (one dimensional) bifurcation diagrams of nonlinear partial differential equations (PDEs). For our illustrations, we considered two benchmark problems, namely (a) the one-dimensional viscous Burgers with both homogeneous (Dirichlet) and non-homogeneous mixed boundary conditions, and, (b) the one and two-dimensional Liouville-Bratu-Gelfand PDEs with homogeneous Dirichlet boundary conditions. For the one-dimensional Burgers and Bratu PDEs, exact analytical solutions are available and used for comparison purposes against the numerical derived solutions. Furthermore, the numerical efficiency (in terms of accuracy and size of the grid) of the proposed numerical machine learning scheme is compared against central finite differences (FD) and Galerkin weighted-residuals finite-element methods (FEM). We show that the proposed ELM numerical method outperforms both FD and FEM methods for medium to large sized grids, while provides equivalent results with the FEM for low to medium sized grids.

Motivation & Objective

• To develop a problem-independent numerical scheme based on Extreme Learning Machines (ELMs) for solving nonlinear PDEs with steep gradients.
• To extend ELMs beyond classification into numerical analysis, particularly for steady-state solutions of PDEs.
• To demonstrate the capability of ELMs to construct one-dimensional bifurcation diagrams via pseudo-arc-length continuation.
• To compare the ELM approach against established methods—central finite differences (FD) and Galerkin finite elements (FEM)—in terms of accuracy and efficiency.
• To validate the method on benchmark problems: one-dimensional viscous Burgers and Liouville–Bratu–Gelfand PDEs with known analytical solutions and turning points.

Proposed method

• The method employs ELMs with single hidden layers using sigmoidal (SF) and radial-basis function (RBF) activation functions to approximate solutions of nonlinear PDEs.
• Collocation is applied at a set of training points across the domain, enforcing the PDE and boundary conditions in a strong form.
• The output weights are computed via a single least-squares solution, avoiding iterative optimization and enabling fast computation.
• Pseudo-arc-length continuation is coupled with Newton–Raphson iterations to trace solution branches past turning points and construct bifurcation diagrams.
• The method is applied to one-dimensional viscous Burgers and Liouville–Bratu–Gelfand PDEs with Dirichlet boundary conditions.
• Numerical performance is evaluated by comparing ELM results with analytical solutions, FD, and FEM across varying grid sizes.

Experimental results

Research questions

• RQ1 Can ELMs effectively approximate steady-state solutions of nonlinear PDEs with sharp gradients, such as those in viscous Burgers and Bratu–Gelfand equations?
• RQ2 How does the ELM-based collocation method compare in accuracy and efficiency to central finite differences and Galerkin finite elements for medium-to-large grid sizes?
• RQ3 Can ELMs be successfully integrated with numerical continuation techniques to trace bifurcation diagrams beyond turning points?
• RQ4 What is the impact of activation function type (sigmoidal vs. RBF) on the convergence and accuracy of the ELM solution for nonlinear PDEs?
• RQ5 Does the ELM approach maintain high accuracy near critical points such as limit points in parameter-dependent PDEs?

Key findings

• For the two-dimensional Bratu problem, the ELM method estimated the turning point at λ ≈ 6.8069 with a relative error of ~0.01% compared to the literature value of 6.808124.
• The ELM with RBF activation achieved the lowest L2 error (1.2×10−10) at N=1600 neurons for the radial Gelfand-Bratu problem with λ=1, outperforming both FD and FEM.
• For the one-dimensional Burgers equation, ELMs matched the analytical solution with L2 error below 10−10 on a 40×40 grid, while FD and FEM showed higher errors.
• The ELM method outperformed both FD and FEM for medium to large grid sizes (N≥196), showing superior accuracy in approximating the turning point location.
• The ELM approach demonstrated robustness in capturing steep gradients near boundaries, where the FD scheme failed due to poor resolution.
• The implementation of ELMs was significantly simpler than FEM, while achieving comparable or better accuracy, especially in high-resolution regimes.