[Paper Review] Efficient distributed matrix-free multigrid methods on locally refined meshes for FEM computations

Peter Münch, Heister, Timo+2 more|arXiv (Cornell University)|Mar 23, 2022

Advanced Numerical Methods in Computational Mathematics4 citations

TL;DR

This paper presents and compares three matrix-free multigrid methods—geometric local smoothing, geometric global coarsening, and polynomial global coarsening—for finite element computations on locally refined meshes using the deal.II library. It demonstrates that global coarsening offers superior parallel scalability due to better load balancing on fine levels, while local smoothing performs better in serial due to lower constraint application costs, despite slightly higher iteration counts.

ABSTRACT

This work studies three multigrid variants for matrix-free finite-element computations on locally refined meshes: geometric local smoothing, geometric global coarsening, and polynomial global coarsening. We have integrated the algorithms into the same framework-the open-source finite-element library deal.II-, which allows us to make fair comparisons regarding their implementation complexity, computational efficiency, and parallel scalability as well as to compare the measurements with theoretically derived performance models. Serial simulations and parallel weak and strong scaling on up to 147,456 CPU cores on 3,072 compute nodes are presented. The results obtained indicate that global coarsening algorithms show a better parallel behavior for comparable smoothers due to the better load balance particularly on the expensive fine levels. In the serial case, the costs of applying hanging-node constraints might be significant, leading to advantages of local smoothing, even though the number of solver iterations needed is slightly higher.

Motivation & Objective

• To evaluate and compare three multigrid variants—geometric local smoothing, geometric global coarsening, and polynomial global coarsening—for matrix-free finite element methods on locally refined meshes.
• To assess implementation complexity, computational efficiency, and parallel scalability of these methods in a unified framework.
• To analyze the performance trade-offs between local and global coarsening strategies in the context of matrix-free FEM with hanging-node constraints.
• To validate the results against theoretical performance models using large-scale serial and parallel simulations up to 147,456 CPU cores.
• To provide open-source, production-ready implementations in the deal.II finite element library for broader adoption and reproducibility.

Proposed method

• Implement all three multigrid variants within the same open-source finite element library (deal.II) to ensure fair, direct comparison.
• Use matrix-free operator evaluations to maximize node-level performance and emphasize communication costs in parallel settings.
• Apply geometric multigrid with local smoothing (smoothing only on the finest level) and global coarsening (coarsening across all levels), as well as polynomial multigrid with global coarsening via reduced polynomial order.
• Use hanging-node constraints to maintain continuity across non-matching refinement levels, with special attention to their computational cost.
• Conduct serial simulations and parallel weak and strong scaling tests on up to 147,456 CPU cores across 3,072 compute nodes.
• Measure solver iterations, time-to-solution, and single-iteration performance to evaluate efficiency and scalability.

Experimental results

Research questions

• RQ1 How do geometric local smoothing, geometric global coarsening, and polynomial global coarsening compare in terms of parallel scalability and load balancing on locally refined meshes?
• RQ2 What is the impact of hanging-node constraint application on serial performance, and does it favor local over global smoothing?
• RQ3 How do the three multigrid variants perform in terms of computational efficiency and iteration count when using matrix-free operators?
• RQ4 To what extent do theoretical performance models align with empirical measurements in large-scale parallel simulations?
• RQ5 Can global coarsening achieve better parallel efficiency than local smoothing despite higher constraint costs on fine levels?

Key findings

• Global coarsening methods exhibit superior parallel scalability due to better load balancing, especially on expensive fine levels, outperforming local smoothing in large-scale parallel runs.
• In serial computations, local smoothing achieves better performance (lower time-to-solution) despite requiring slightly more iterations, due to reduced overhead from hanging-node constraint application.
• For p = 4 and L = 10, global coarsening achieves a time-to-solution of 3.4e-1 seconds on 768 processes (16 nodes), while local smoothing takes 3.0e-1 seconds, indicating a small but measurable serial advantage for local smoothing.
• On 24,576 processes (512 nodes), global coarsening maintains stable performance across varying cell weight factors near hanging nodes, with time-to-solution around 1.0e-1 seconds for p = 4 and L = 10.
• Polynomial global coarsening shows comparable performance to geometric global coarsening, with both outperforming local smoothing in parallel scaling, particularly at high core counts.
• AMG as a coarse-grid solver in p-multigrid shows high iteration counts and long solve times, especially for higher V-cycle counts, making it less efficient than the proposed multigrid variants in this setting.