[Paper Review] Accelerating high order discontinuous Galerkin solvers using neural networks: 3D compressible Navier-Stokes equations

Fernando Manrique de Lara, Esteban Ferrer|arXiv (Cornell University)|Jul 23, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes accelerating high-order discontinuous Galerkin (DG) simulations of 3D compressible Navier-Stokes equations by using a deep neural network to learn a corrective forcing term for low-order DG simulations. Trained on high-order data over a short time window, the neural network enables simulations with polynomial order 3 to achieve accuracy comparable to order 6, reducing computational cost by 4–5× while maintaining accuracy across laminar, transitional, and turbulent regimes at Reynolds numbers 30, 200, and 1600.

ABSTRACT

We propose to accelerate a high order discontinuous Galerkin solver using neural networks. We include a corrective forcing to a low polynomial order simulation to enhance its accuracy. The forcing is obtained by training a deep fully connected neural network, using a high polynomial order simulation but only for a short time frame. With this corrective forcing, we can run the low polynomial order simulation faster (with large time steps and low cost per time step) while improving its accuracy. We explored this idea for a 1D Burgers' equation in (Marique and Ferrer, CAF 2022), and we have extended this work to the 3D Navier-Stokes equations, with and without a Large Eddy Simulation closure model. We test the methodology with the turbulent Taylor Green Vortex case and for various Reynolds numbers (30, 200 and 1600). In addition, the Taylor Green Vortex evolves with time and covers laminar, transitional, and turbulent regimes, as time progresses. The proposed methodology proves to be applicable to a variety of flows and regimes. The results show that the corrective forcing is effective in all Reynolds numbers and time frames (excluding the initial flow development). We can train the corrective forcing with a polynomial order of 8, to increase the accuracy of simulations from a polynomial order 3 to 6, when correcting outside the training time frame. The low order correct solution is 4 to 5 times faster than a simulation with comparable accuracy (polynomial order 6). Additionally, we explore changes in the hyperparameters and use transfer learning to speed up the training. We observe that it is not useful to train a corrective forcing using a different flow condition. However, an already trained corrective forcing can be used to initialise a new training (at the correct flow conditions) to obtain an effective forcing with only a few training iterations.

Motivation & Objective

• To accelerate high-order discontinuous Galerkin (DG) solvers for 3D compressible Navier-Stokes equations without sacrificing accuracy.
• To reduce computational cost by using low polynomial order simulations with a neural network–learned corrective forcing term.
• To extend a prior 1D Burgers’ equation methodology to 3D turbulent flows, including transitional and fully turbulent regimes.
• To evaluate the effectiveness of the corrective forcing across varying Reynolds numbers and flow regimes.
• To investigate hyperparameter sensitivity and transfer learning for faster training of corrective networks.

Proposed method

• Train a deep fully connected neural network on high-order DG simulation data over a short time frame to learn a corrective forcing term.
• Apply the trained neural network forcing to a low-order DG simulation (polynomial order 3) to improve its accuracy toward high-order (order 6) solutions.
• Use the corrective forcing to enable larger time steps and reduce degrees of freedom, accelerating the low-order simulation.
• Implement a post-processing filtering step to improve solution quality and reduce numerical oscillations.
• Model error dynamics using linearized equations based on the Jacobian of the governing equations and the neural network’s sensitivity to estimate error propagation.
• Utilize transfer learning by initializing a new network with a pre-trained forcing model at new flow conditions to reduce training iterations.

Experimental results

Research questions

• RQ1 Can a neural network–learned corrective forcing effectively improve the accuracy of low-order DG simulations for 3D compressible Navier-Stokes equations?
• RQ2 To what extent does the corrective forcing maintain accuracy beyond the training time window and across different Reynolds numbers?
• RQ3 How does the method perform across varying flow regimes, including laminar, transitional, and turbulent states?
• RQ4 Can transfer learning reduce training time for corrective forcing networks when applied to new flow conditions?
• RQ5 What is the impact of hyperparameter choices on the performance and generalization of the corrective forcing network?

Key findings

• The corrective forcing trained on high-order data (polynomial order 8) successfully enhances low-order simulations (order 3) to achieve accuracy comparable to order 6 simulations, even outside the training time window.
• The low-order corrected simulation runs 4 to 5 times faster than a standard order 6 DG simulation with similar accuracy.
• The method is effective across all tested Reynolds numbers (30, 200, and 1600), including laminar, transitional, and turbulent flow regimes.
• Transfer learning enables effective training with only a few iterations when initializing from a pre-trained model at the correct flow condition.
• Training the corrective forcing on different flow conditions is not useful, but reusing a pre-trained model as a starting point significantly accelerates convergence.
• Error analysis shows that the neural network forcing reduces the dominant error source (missing subgrid-scale forcing) and stabilizes the solution, with error growth bounded by effective coefficients derived from the system’s Jacobian and NN sensitivity.