[Paper Review] A parallel implementation of a diagonalization-based parallel-in-time integrator.

Gayatri Caklovic, Robert Speck+1 more|arXiv (Cornell University)|Mar 23, 2021

Numerical methods for differential equations29 references4 citations

TL;DR

This paper presents an open-source, space- and doubly time-parallel implementation of a diagonalization-based parallel-in-time integrator using adaptive $α$-circulant preconditioning. It extends the method with arbitrary-order collocation for enhanced time parallelism and demonstrates improved convergence and efficiency through runtime parameter adaptation, validated on two test problems with strong performance scaling.

ABSTRACT

We present and analyze a parallel implementation of a parallel-in-time method based on $\alpha$-circulant preconditioned Richardson iterations. While there are a lot of papers exploring this new class of single-level, time-parallel integrators from many perspectives, performance results of actual parallel runs are still missing. This leaves a critical gap, because the efficiency and applicability heavily rely on the actual parallel performance, with only limited guidance from theoretical considerations. Also, many challenges like selecting good parameters, finding suitable communication strategies, and performing a fair comparison to sequential time-stepping methods can be easily missed. In this paper, we first extend the original idea by using a collocation method of arbitrary order, which adds another level of parallelization in time. We derive an adaptive strategy to select a new $\alpha$-circulant preconditioner for each iteration during runtime for balancing convergence rates, round-off errors and inexactness in the individual time-steps. After addressing these more theoretical challenges, we present an open-source space- and doubly-time-parallel implementation and evaluate its performance for two different test problems.

Motivation & Objective

• To close the gap in empirical performance evaluation of parallel-in-time methods, especially for single-level, diagonalization-based integrators.
• To address practical challenges in parallel-in-time computing, including parameter selection, communication strategies, and fair comparison with sequential time-stepping.
• To extend the original method with a high-order collocation approach to introduce an additional level of time parallelism.
• To develop and implement an adaptive strategy for selecting $α$-circulant preconditioners at each iteration to balance convergence, round-off errors, and inexactness.
• To provide a production-ready, open-source implementation for reproducible performance evaluation and broader applicability.

Proposed method

• The method employs a diagonalization-based parallel-in-time integrator using $α$-circulant preconditioned Richardson iterations for solving time-implicit systems.
• It integrates a collocation method of arbitrary order to enhance time-parallelism, enabling multiple time steps to be processed simultaneously.
• An adaptive strategy dynamically selects the $α$-circulant preconditioner at each iteration based on convergence behavior, round-off considerations, and inexactness in time-step solves.
• The preconditioner adaptation balances convergence speed, numerical accuracy, and computational cost during runtime.
• The implementation is fully parallel in space and time, leveraging both MPI and OpenMP for distributed and shared memory parallelism.
• The framework is open-sourced to ensure reproducibility and facilitate community adoption and benchmarking.

Experimental results

Research questions

• RQ1 How does the performance of the diagonalization-based parallel-in-time method scale with increasing problem size and number of processors?
• RQ2 What is the impact of using high-order collocation on the convergence and parallel efficiency of the time-parallel integrator?
• RQ3 How effective is the adaptive selection of $α$-circulant preconditioners in balancing convergence, round-off errors, and inexactness during runtime?
• RQ4 How does the method compare to traditional sequential time-stepping in terms of wall-clock time and scalability?
• RQ5 What communication strategies and parameter choices yield optimal performance in practice?

Key findings

• The adaptive $α$-circulant preconditioning strategy significantly improves convergence rates while maintaining numerical stability and reducing round-off errors.
• The integration of high-order collocation enables a second level of time parallelism, enhancing overall efficiency and scalability.
• The open-source implementation demonstrates strong scaling behavior on large-scale systems, with performance gains increasing with problem size.
• The method achieves meaningful speedups over sequential time-stepping, particularly for large time-domain problems.
• The performance evaluation reveals that communication overhead and parameter tuning are critical factors affecting real-world efficiency.
• The results confirm that theoretical convergence properties alone are insufficient for predicting real-world performance, underscoring the need for empirical evaluation.