[Paper Review] PAMPAC: A Parallel Adaptive Method for Pseudo-Arclength Continuation

D. A. Aruliah, Lennaert van Veen+1 more|arXiv (Cornell University)|Dec 12, 2013

Parallel Computing and Optimization Techniques14 references3 citations

TL;DR

PAMPAC is a parallel adaptive algorithm for pseudo-arclength continuation that accelerates convergence in high-dimensional nonlinear systems by concurrently computing multiple predictor-corrector sequences with varying step-sizes. By reusing intermediate corrector iterates to seed new predictions and leveraging redundant computation, it achieves up to a 3x speed-up on a 39-CPU cluster, significantly reducing time wasted on failed correction steps in costly Newton-Krylov solves.

ABSTRACT

Pseudo-arclength continuation is a well-established method for generating a numerical curve approximating the solution of an underdetermined system of nonlinear equations. It is an inherently sequential predictor-corrector method in which new approximate solutions are extrapolated from previously converged results and then iteratively refined. Convergence of the iterative corrections is guaranteed only for sufficiently small prediction steps. In high-dimensional systems, corrector steps are extremely costly to compute and the prediction step-length must be adapted carefully to avoid failed steps or unnecessarily slow progress. We describe a parallel method for adapting the step-length employing several predictor-corrector sequences of different step lengths computed concurrently. In addition, the algorithm permits intermediate results of unconverged correction sequences to seed new predictions. This strategy results in an aggressive optimization of the step length at the cost of redundancy in the concurrent computation. We present two examples of convoluted solution curves of high-dimensional systems showing that speed-up by a factor of two can be attained on a multi-core CPU while a factor of three is attainable on a small cluster.

Motivation & Objective

• To address the performance bottleneck in pseudo-arclength continuation caused by costly, failed corrector iterations in high-dimensional systems.
• To reduce wall-clock time in numerical continuation by minimizing wasted computation on non-convergent predictor steps.
• To enable aggressive step-size adaptation through concurrent computation of multiple predictor-corrector sequences with varying step-sizes.
• To exploit intermediate corrector iterates as seeds for new predictions, improving efficiency without waiting for convergence.
• To demonstrate practical speed-up in real-world PDE-based continuation problems with expensive linear solves.

Proposed method

• The algorithm computes multiple predictor-corrector sequences in parallel, each with a different step-size, using a tree-structured approach to manage concurrent computations.
• It employs a recursive strategy where intermediate, unconverged corrector iterates from one sequence can seed new predictor steps in other sequences.
• Step-size adaptation is guided by convergence behavior: successful sequences inform future step-size choices, while failed ones trigger reduction.
• The method is designed for systems where each corrector iteration is computationally expensive (e.g., Newton-Krylov solves with GMRES), and communication cost is negligible.
• It uses a pseudo-arclength parameterization to maintain curve-following stability and ensures predictor steps remain close to the solution manifold.
• The algorithm is implemented as a software library (PAMPAC) with support for multi-core CPUs and small clusters, using MPI for inter-process communication.

Experimental results

Research questions

• RQ1 Can parallel execution of multiple predictor-corrector sequences with varying step-sizes reduce the wall-clock time of pseudo-arclength continuation in high-dimensional systems?
• RQ2 To what extent can intermediate, unconverged corrector iterates be reused to accelerate convergence and avoid redundant computation?
• RQ3 How does the performance of the parallel adaptive method scale with the number of processors in realistic PDE-based continuation problems?
• RQ4 What is the achievable speed-up in wall-clock time when applying PAMPAC to systems with expensive corrector steps, such as Newton-Krylov solves with GMRES?
• RQ5 Does the method maintain robustness and stability in the presence of high-curvature solution curves where optimal step-sizes vary significantly?

Key findings

• PAMPAC achieved a speed-up of up to a factor of three on a 39-CPU cluster when solving a high-symmetric flow problem with expensive Newton-Krylov corrector steps.
• A speed-up of up to a factor of two was achieved using only three CPUs, demonstrating effectiveness even on small-scale parallel systems.
• The method reduced the time wasted on failed corrector steps by reusing intermediate iterates and exploring multiple step-sizes in parallel, avoiding sequential rejection cycles.
• In a serial implementation with naive step-size control, 15 of 55 correction steps were rejected (27% wasted time), whereas PAMPAC significantly reduced this overhead through proactive adaptation.
• The algorithm’s performance is most effective when corrector iterations are expensive relative to communication cost and when the solution curve exhibits significant curvature variation.
• The tree-structured parallelization strategy with dynamic step-size adaptation proved robust and scalable across different configurations, with tree width having a greater impact on performance than depth.