[Paper Review] Global finite element matrix construction based on a CPU-GPU implementation

Francisco Javier Ramírez-Gil, Marcos de Sales Guerra Tsuzuki+1 more|arXiv (Cornell University)|Jan 20, 2015

Matrix Theory and Algorithms4 references4 citations

TL;DR

This paper proposes a CPU-GPU heterogeneous computing approach to accelerate global finite element matrix construction for 3D unstructured meshes. By offloading parallel numerical integration to the GPU and using the CPU for serial matrix assembly, the method achieves up to 126x speedup over CPU-only computation and enables construction of matrices with up to 27 million elements using only 2GB GPU memory.

ABSTRACT

The finite element method (FEM) has several computational steps to numerically solve a particular problem, to which many efforts have been directed to accelerate the solution stage of the linear system of equations. However, the finite element matrix construction, which is also time-consuming for unstructured meshes, has been less investigated. The generation of the global finite element matrix is performed in two steps, computing the local matrices by numerical integration and assembling them into a global system, which has traditionally been done in serial computing. This work presents a fast technique to construct the global finite element matrix that arises by solving the Poisson's equation in a three-dimensional domain. The proposed methodology consists in computing the numerical integration, due to its intrinsic parallel opportunities, in the graphics processing unit (GPU) and computing the matrix assembly, due to its intrinsic serial operations, in the central processing unit (CPU). In the numerical integration, only the lower triangular part of each local stiffness matrix is computed thanks to its symmetry, which saves GPU memory and computing time. As a result of symmetry, the global sparse matrix also contains non-zero elements only in its lower triangular part, which reduces the assembly operations and memory usage. This methodology allows generating the global sparse matrix from any unstructured finite element mesh size on GPUs with little memory capacity, only limited by the CPU memory.

Motivation & Objective

• Address the performance bottleneck in global finite element matrix construction, which remains under-investigated despite advances in solving the resulting linear systems.
• Overcome the limitation of GPU memory in handling large unstructured 3D meshes by combining CPU and GPU workloads effectively.
• Develop a scalable and memory-efficient method for constructing large sparse global matrices using heterogeneous computing with limited GPU resources.
• Enable high-performance FEM matrix generation on standard personal computers equipped with low-memory GPUs by leveraging CPU memory for scalability.

Proposed method

• Decompose the finite element matrix construction into two stages: local matrix computation via numerical integration and global matrix assembly.
• Implement numerical integration—highly parallel—on the GPU, computing only the lower triangular part of each element stiffness matrix to save memory and time.
• Use the CPU for matrix assembly, which is inherently serial and memory-intensive, to avoid GPU memory bottlenecks.
• Exploit matrix symmetry: since both local and global stiffness matrices are symmetric, only the lower triangular parts are stored and assembled, reducing memory usage and operations.
• Use asynchronous kernel launches to overlap GPU computation (numerical integration) with CPU computation (assembly), improving overall throughput.
• Partition large meshes into groups based on available GPU memory, enabling construction of matrices larger than GPU memory by relying on CPU memory.

Experimental results

Research questions

• RQ1 Can a CPU-GPU heterogeneous architecture significantly accelerate the construction of large global finite element matrices for unstructured 3D meshes?
• RQ2 How can matrix symmetry be leveraged to reduce memory usage and computational cost in GPU-based finite element matrix generation?
• RQ3 To what extent can the method scale to large matrices when GPU memory is limited, and what role does CPU memory play in this scaling?
• RQ4 What performance gains are achievable by offloading numerical integration to the GPU while keeping the serial assembly on the CPU?
• RQ5 Can this approach be generalized beyond the Poisson equation to other PDEs and matrix types (e.g., mass matrices) in the finite element method?

Key findings

• The proposed method achieved a maximum speedup of 126x and a mean speedup of 121x over a serial CPU implementation for numerical integration, demonstrating high GPU utilization.
• The numerical integration routine consumed 62–82% of total matrix generation time, while the assembly routine consumed only 17–38%, indicating that numerical integration is the primary performance bottleneck.
• The sparse matrix assembly function (sparse_create) was highly efficient, completing in under 20 seconds even for 27 million elements—only one-fifth the time of the numerical integration phase.
• The method successfully constructed a global sparse matrix with 27.27 million nodes and 27 million 3D elements using only 2GB of GPU memory, limited only by CPU memory capacity.
• Asynchronous execution of GPU kernels allowed overlapping of numerical integration and matrix assembly, improving resource utilization and reducing idle time.
• The approach is generalizable: it can be applied to construct stiffness matrices for static problems and mass matrices for dynamic problems, as well as other PDEs using the FEM.