[论文解读] Numerical stability analysis of the class of communication hiding pipelined Conjugate Gradient methods.

Siegfried Cools|arXiv (Cornell University)|Apr 9, 2018

Matrix Theory and Algorithms参考文献 32被引用 1

一句话总结

本文分析了流水线共轭梯度（CG）方法的数值稳定性，该方法通过将全局规约为计算与本地计算重叠，减少了高性能计算（HPC）中的通信延迟。论文推导了递归计算变量的误差间隙表达式，展示了舍入误差如何影响收敛性和精度，并为可扩展至百亿亿次计算的求解器提供了校正这些影响的框架。

ABSTRACT

Krylov subspace methods are widely known as efficient algebraic methods for solving linear systems. However, on massively parallel hardware their performance is typically limited by communication latency rather than floating point performance. With HPC hardware advancing towards the exascale regime the gap between computation (i.e. flops) and communication (i.e. internode communication, as well as data movement within the memory hierarchy) keeps steadily increasing, imposing the need for scalable alternatives to traditional Krylov subspace methods. One such approach are pipelined Krylov subspace methods, which reduce the number of global synchronization points and overlap global communication latency with local arithmetic operations, thus `hiding' the global reduction phases behind useful computations. To obtain this overlap the algorithm is reformulated by introducing a number of auxiliary vector quantities, which are computed using additional recurrence relations. Although pipelined Krylov subspace methods are equivalent to traditional Krylov subspace methods in exact arithmetic, the behavior of local rounding errors induced by the multi-term recurrence relations in finite precision may in practice affect convergence significantly. This numerical stability study aims to characterize the effect of local rounding errors in various pipelined versions of the popular Conjugate Gradient method. We derive expressions for the gaps between the true and (recursively) computed variables that are used to update the search directions in the different CG variants. Furthermore, we show how these results can be used to analyze and correct the effect of local rounding error propagation on the maximal attainable accuracy of pipelined CG methods. The analysis in this work is supplemented by various numerical experiments that demonstrate the numerical stability of the pipelined CG methods.

研究动机与目标

• 解决百亿亿次HPC系统中浮点运算与通信延迟之间日益扩大的性能差距。
• 研究流水线CG方法中多术语递推关系产生的局部舍入误差如何影响收敛性和精度。
• 表征由于有限精度算术运算，流水线CG变体中真实变量与计算变量之间的偏差。
• 建立一个理论框架，以分析和校正舍入误差传播对最大可实现精度的影响。
• 通过数值实验验证理论发现，展示不同流水线CG实现中的稳定性。

提出的方法

• 推导多个流水线CG变体中用于搜索方向更新的真正变量与递归计算变量之间间隙的解析表达式。
• 对流水线Krylov方法固有的多术语递推关系中局部舍入误差的传播进行建模。
• 利用精确算术等价性，识别仅由有限精度计算引起的偏差。
• 基于推导出的间隙表达式，制定误差校正机制，以提高最大可实现精度。
• 在数值实验中实现并测试流水线CG变体，以验证理论预测的稳定性与收敛行为。

实验结果

研究问题

• RQ1 流水线CG方法中的多术语递推关系在有限精度下如何放大局部舍入误差？
• RQ2 由于舍入误差，流水线CG中真实变量与计算变量之间的定量关系是什么？
• RQ3 这些舍入误差在多大程度上会降低流水线CG的收敛性与最大可实现精度？
• RQ4 推导出的误差间隙表达式能否用于预测并校正有限精度算术对解精度的影响？
• RQ5 不同流水线CG变体的数值结果与理论预测相比如何？

主要发现

• 本文推导出流水线CG方法中真正变量与递归计算变量之间间隙的显式表达式，量化了有限精度算术的影响。
• 流水线CG中的舍入误差通过递推关系传播，可能显著降低收敛性与最大可实现精度。
• 推导出的误差间隙表达式可识别算法中误差累积最显著的关键位置。
• 数值实验验证了理论预测，表明在考虑误差传播后，收敛行为保持稳定。
• 该分析为设计更鲁棒的流水线Krylov求解器提供了基础，可通过基于推导间隙模型的误差校正策略实现。