[Paper Review] Resilience for Exascale Enabled Multigrid Methods

Markus Huber, Björn Gmeiner+2 more|arXiv (Cornell University)|Jan 29, 2015

Parallel Computing and Optimization Techniques25 references5 citations

TL;DR

This paper proposes an algorithm-based fault tolerance (ABFT) strategy for parallel geometric multigrid solvers in exascale computing, where faults are recovered by solving local subproblems with Dirichlet boundary conditions using multigrid cycles. The method achieves convergence comparable to checkpoint-restart with minimal I/O overhead, demonstrating that local multigrid recovery outperforms smoothing and Krylov solvers in accuracy and efficiency.

ABSTRACT

With the increasing number of components and further miniaturization the mean time between faults in supercomputers will decrease. System level fault tolerance techniques are expensive and cost energy, since they are often based on redundancy. Also classical check-point-restart techniques reach their limits when the time for storing the system state to backup memory becomes excessive. Therefore, algorithm-based fault tolerance mechanisms can become an attractive alternative. This article investigates the solution process for elliptic partial differential equations that are discretized by finite elements. Faults that occur in the parallel geometric multigrid solver are studied in various model scenarios. In a standard domain partitioning approach, the impact of a failure of a core or a node will affect one or several subdomains. Different strategies are developed to compensate the effect of such a failure algorithmically. The recovery is achieved by solving a local subproblem with Dirichlet boundary conditions using local multigrid cycling algorithms. Additionally, we propose a superman strategy where extra compute power is employed to minimize the time of the recovery process.

Motivation & Objective

• Address the growing challenge of hardware failures in exascale HPC systems, where millions of nodes increase fault probability.
• Overcome the limitations of traditional checkpoint-restart techniques, which incur high I/O and energy costs due to frequent state saving.
• Develop an algorithm-based fault tolerance (ABFT) approach that enables fault recovery without relying on system-level redundancy or checkpointing.
• Minimize solution time delay and computational overhead during recovery while ensuring convergence after node or core failures.
• Demonstrate that local multigrid cycling is more effective than smoothing or Krylov methods for fault recovery in geometric multigrid solvers.

Proposed method

• Model the failure scenario as a core or node failure in a parallel geometric multigrid solver, leading to loss of solution values in a subdomain.
• Recover lost values by solving a local subproblem with Dirichlet boundary conditions on the affected subdomain using local multigrid cycles (V-, W-, F-cycles).
• Introduce a 'superman' strategy that leverages extra compute resources to accelerate local recovery through parallelization.
• Use the Cycle Advantage (CCR) metric to compare recovery efficiency against full checkpoint-restart (CCR) in terms of number of multigrid cycles required.
• Implement and evaluate the recovery strategies within the hierarchical hybrid multigrid (HHG) framework, using finite element discretizations of the Poisson equation.
• Compare recovery performance across different smoothers (Gauss-Seidel, Jacobi), PCG iterations, and multigrid cycles under varying fault timing.

Experimental results

Research questions

• RQ1 Can local multigrid cycling effectively recover from node or core failures in a parallel geometric multigrid solver without checkpointing?
• RQ2 How does the timing of a fault (early vs. late in the solution process) affect the required recovery effort and convergence rate?
• RQ3 What is the relative performance of different recovery strategies—smoothers, PCG, and multigrid cycles—in achieving accuracy comparable to checkpoint-restart?
• RQ4 Can the 'superman' strategy significantly reduce recovery time by exploiting additional parallelism?
• RQ5 How does the Cycle Advantage metric quantify the efficiency of recovery strategies relative to full checkpoint-restart?

Key findings

• For a fault occurring after 5 iterations, 4 V-cycles, 3 W-cycles, or 3 F-cycles achieve a condition number (κ) of 4.164, matching the accuracy of the complete checkpoint-restart (CCR) strategy.
• When a fault occurs after 7 iterations, 5 W- or F-cycles are required to reach κ ≈ 6.445, matching the CCR result, while smoother and PCG strategies show only marginal improvement.
• For a fault after 11 iterations, more than 5 W- or F-cycles are needed to match the CCR performance (κ ≈ 10.999), indicating increasing recovery cost with delayed fault detection.
• W-cycles and F-cycles yield nearly identical recovery performance, but F-cycles are preferred due to lower computational cost.
• Smoothing and PCG-based recovery strategies provide only marginal improvements over baseline methods and fail to match the accuracy of multigrid cycles.
• The CCR metric confirms that multigrid cycles are the only recovery method that can achieve CCR-like accuracy with minimal I/O and computational overhead, making them ideal for exascale resilience.