[Paper Review] Simulating Turbulence Using the Astrophysical Discontinuous Galerkin Code TENET

Andreas Bauer, Kevin Schaal+4 more|arXiv (Cornell University)|Feb 29, 2016

Computer Graphics and Visualization Techniques4 citations

TL;DR

This paper introduces TENET, a new astrophysical discontinuous Galerkin (DG) code designed for simulating subsonic turbulence in cosmological contexts. Using high-order DG discretization with compact stencils and efficient parallelization, TENET achieves comparable accuracy to traditional finite volume methods with 1.8 times fewer degrees of freedom and runs 1.5 times faster, demonstrating strong computational efficiency and promise for exascale astrophysical simulations.

ABSTRACT

In astrophysics, the two main methods traditionally in use for solving the Euler equations of ideal fluid dynamics are smoothed particle hydrodynamics and finite volume discretization on a stationary mesh. However, the goal to efficiently make use of future exascale machines with their ever higher degree of parallel concurrency motivates the search for more efficient and more accurate techniques for computing hydrodynamics. Discontinuous Galerkin (DG) methods represent a promising class of methods in this regard, as they can be straightforwardly extended to arbitrarily high order while requiring only small stencils. Especially for applications involving comparatively smooth problems, higher-order approaches promise significant gains in computational speed for reaching a desired target accuracy. Here, we introduce our new astrophysical DG code TENET designed for applications in cosmology, and discuss our first results for 3D simulations of subsonic turbulence. We show that our new DG implementation provides accurate results for subsonic turbulence, at considerably reduced computational cost compared with traditional finite volume methods. In particular, we find that DG needs about 1.8 times fewer degrees of freedom to achieve the same accuracy and at the same time is more than 1.5 times faster, confirming its substantial promise for astrophysical applications.

Motivation & Objective

• To develop a high-order, efficient, and scalable hydrodynamics solver for astrophysical simulations using discontinuous Galerkin (DG) methods.
• To evaluate the performance and accuracy of DG methods in simulating subsonic turbulence, a benchmark problem in computational astrophysics.
• To compare DG-based simulations in TENET with traditional second-order finite volume schemes in terms of computational cost, accuracy, and scalability.
• To assess the potential of DG methods for future exascale computing in astrophysics, particularly for smooth, turbulent flows.
• To investigate whether DG can achieve better computational efficiency than finite volume methods while maintaining or improving accuracy.

Proposed method

• The code uses a discontinuous Galerkin (DG) discretization of the Euler equations, representing the solution via high-order polynomial basis functions within each element.
• The DG method employs weak formulations of the equations, solved via numerical fluxes at element interfaces, enabling high-order accuracy and local conservation.
• The method uses a modal or nodal basis representation within elements, allowing for arbitrary order of accuracy with only direct neighbor communication.
• Time integration is performed using a strong-stability-preserving Runge-Kutta scheme to maintain stability and accuracy.
• The code is implemented on a structured Cartesian mesh with support for adaptive mesh refinement (AMR), enabling efficient resolution of turbulent structures.
• Parallelization is achieved via domain decomposition with MPI, and the method shows excellent strong scaling up to 4096 cores on the SUPERMUC system.

Experimental results

Research questions

• RQ1 Can a high-order discontinuous Galerkin code like TENET achieve comparable accuracy to traditional second-order finite volume schemes in simulating subsonic turbulence?
• RQ2 How does the computational cost of DG methods compare to finite volume methods when achieving the same level of accuracy in turbulent flow simulations?
• RQ3 To what extent does the DG method reduce the number of degrees of freedom required to achieve a given accuracy compared to finite volume schemes?
• RQ4 How does the performance of the DG method scale with increasing resolution and number of processors in a parallel environment?
• RQ5 Does the DG approach offer advantages in conservation properties (e.g., angular momentum or div-B constraints) over traditional finite volume methods in smooth astrophysical flows?

Key findings

• The DG implementation in TENET achieves the same accuracy in subsonic turbulence simulations with approximately 1.8 times fewer degrees of freedom than the second-order finite volume method.
• For simulations with comparable turbulence power spectra, the DG run at third order is 1.53 times faster than the corresponding second-order finite volume run at the highest resolution studied.
• The performance advantage of DG increases with resolution: at the highest resolution, the DG_128_3 run is 1.53 times faster than the FV_512_2 run, approaching a 1.86-fold reduction in degrees of freedom.
• The DG method exhibits excellent strong parallel scaling up to 4096 cores on SUPERMUC, indicating high efficiency on large-scale distributed systems.
• The DG method manifestsly conserves angular momentum in smooth flow regions, a property not inherently satisfied by standard finite volume schemes.
• The DG approach enables robust enforcement of the div B = 0 constraint in MHD through basis function-level constraints, offering a pathway to improved MHD simulations.