[Paper Review] A generic finite element framework on parallel tree-based adaptive meshes

Santiago Badia, Alberto F. Martı́n+2 more|arXiv (Cornell University)|Jul 8, 2019

Advanced Numerical Methods in Computational MathematicsEngineering23 references20 citations

TL;DR

This paper presents a formally verified, parallel, distributed-memory finite element framework for h-adaptive simulations on tree-based adaptive meshes using a two-layered mesh representation: a lightweight forest-of-octrees for mesh management and a rich, conforming finite element-compatible mesh layer. The framework ensures correctness of hanging node constraints and parallel assembly via rigorous mathematical proofs, achieving strong scaling up to 32.2K cores and 2–3× performance improvements over deal.II on adaptive Poisson and Maxwell problems.

ABSTRACT

In this work we formally derive and prove the correctness of the algorithms and data structures in a parallel, distributed-memory, generic finite element framework that supports h-adaptivity on computational domains represented as forest-of-trees. The framework is grounded on a rich representation of the adaptive mesh suitable for generic finite elements that is built on top of a low-level, light-weight forest-of-trees data structure handled by a specialized, highly parallel adaptive meshing engine, for which we have identified the requirements it must fulfill to be coupled into our framework. Atop this two-layered mesh representation, we build the rest of data structures required for the numerical integration and assembly of the discrete system of linear equations. We consider algorithms that are suitable for both subassembled and fully-assembled distributed data layouts of linear system matrices. The proposed framework has been implemented within the FEMPAR scientific software library, using p4est as a practical forest-of-octrees demonstrator. A strong scaling study of this implementation when applied to Poisson and Maxwell problems reveals remarkable scalability up to 32.2K CPU cores and 482.2M degrees of freedom. Besides, a comparative performance study of FEMPAR and the state-of-the-art deal.ii finite element software shows at least comparative performance, and at most factor 2-3 improvements in the h-adaptive approximation of a Poisson problem with first- and second-order Lagrangian finite elements, respectively.

Motivation & Objective

• To develop a generic, scalable, and correct finite element framework for parallel h-adaptive simulations on tree-based adaptive meshes.
• To formally prove the correctness of parallel algorithms for handling hanging nodes and constraints in distributed-memory environments.
• To enable efficient, scalable assembly of linear systems on adaptive, non-conforming meshes using a two-layered mesh abstraction.
• To achieve high performance and strong scaling on large-scale problems using a modular, extensible software design.
• To provide a foundation for supporting diverse cell topologies, refinement rules, and finite element types in a unified framework.

Proposed method

• Employ a two-layered mesh abstraction: a low-level, lightweight forest-of-octrees (via p4est) for mesh management and a high-level, conforming mesh representation for finite element operations.
• Use Space-Filling Curves (SFCs), specifically Morton indices, to enable efficient traversal, partitioning, and neighbor-finding in adaptive tree meshes.
• Formalize the mesh data structures and algorithms using mathematical propositions and proofs to guarantee correctness of hanging node constraints and ghost cell dependencies.
• Design data structures and algorithms for both subassembled and fully-assembled matrix layouts to support non-overlapping domain decomposition solvers.
• Implement the framework within the FEMPAR library, using p4est as the underlying forest-of-octrees engine.
• Ensure that all constraints on hanging degrees of freedom are computed locally using only a single layer of ghost cells, validated via rigorous proofs of dependency locality.

Experimental results

Research questions

• RQ1 Under what conditions can hanging node constraints be correctly computed in a parallel, distributed-memory environment with only a single layer of ghost cells?
• RQ2 How can a generic finite element framework be designed to support arbitrary cell topologies and refinement rules while ensuring correctness on tree-based adaptive meshes?
• RQ3 What mathematical guarantees can be provided for the locality of constraint dependencies across processor boundaries in adaptive, non-conforming meshes?
• RQ4 How does the proposed framework achieve strong scalability and performance comparable or superior to state-of-the-art finite element software like deal.II?
• RQ5 What are the necessary and sufficient conditions on the mesh refinement and balancing rules to ensure that constraint dependencies do not exceed the single-layer ghost cell boundary?

Key findings

• The framework achieves strong scaling up to 32,200 CPU cores and 482.2 million degrees of freedom on Poisson and Maxwell problems.
• The implementation in FEMPAR shows at least comparable, and up to 2–3 times better performance than deal.II for h-adaptive solutions using first- and second-order Lagrangian finite elements.
• The mathematical proofs formally establish that hanging node constraints depend only on locally accessible degrees of freedom when using a 2:1 k-balanced mesh and single-layer ghost cells.
• The framework ensures correctness of parallel assembly by proving that constraint dependencies do not exceed the single-layer ghost cell boundary under the 2:1 k-balance condition.
• The two-layered mesh abstraction—separating low-level mesh management from high-level finite element data structures—enables generality, correctness, and high performance.
• The approach is extensible to arbitrary polytopic cells and refinement rules, as demonstrated by the theoretical foundation and implementation in t8code and FEMPAR.