[论文解读] A Makespan Lower Bound for the Scheduling of the Tiled Cholesky Factorization based on ALAP scheduling

Willy Quach, Julien Langou|arXiv (Cornell University)|Oct 17, 2015

Parallel Computing and Optimization Techniques参考文献 15被引用 4

一句话总结

本文為在同质、无通信开销模型下，基于ALAP（尽可能推迟）调度的分块Cholesky分解，建立了紧致的完成时间（makespan）理论下界，证明至少需要0.185t²个处理单元才能实现关键路径长度为9t−10的目标。此外，本文提出一种基于ALAP的调度策略，仅需少于0.25t² + 0.16t + 3个处理单元即可达到该下界，显著优于朴素下界与实际启发式方法。

ABSTRACT

Due to the advent of multicore architectures and massive parallelism, the tiled Cholesky factorization algorithm has recently received plenty of attention and is often referenced by practitioners as a case study. It is also implemented in mainstream dense linear algebra libraries. However, we note that theoretical study of the parallelism of this algorithm is currently lacking. In this paper, we present new theoretical results about the tiled Cholesky factorization in the context of a parallel homogeneous model without communication costs. We use standard flop-based weights for the tasks. For a $t$-by-$t$ matrix, we know that the critical path of the tiled Cholesky algorithm is $9t-10$ and that the weight of all tasks is $t^3$. In this context, we prove that no schedule with less than $0.185 t^2$ processing units can finish in a time less than the critical path. In perspective, a naive bound gives $0.11 t^2.$ We then give a schedule which needs less than $0.25 t^2+0.16t+3$ processing units to complete in the time of the critical path. In perspective, a naive schedule gives $0.50 t^2.$ In addition, given a fixed number of processing units, $p$, we give a lower bound on the execution time as follows: $$\max( \frac{t^{3}}{p}, \frac{t^{3}}{p} - 3\frac{t^2}{p} + 6\sqrt{2p} - 7 , 9t-10).$$ The interest of the latter formula lies in the middle term. Our results stem from the observation that the tiled Cholesky factorization is much better behaved when we schedule it with an ALAP (As Late As Possible) heuristic than an ASAP (As Soon As Possible) heuristic. We also provide scheduling heuristics which match closely the lower bound on execution time. We believe that our theoretical results will help practical scheduling studies. Indeed, our results enable to better characterize the quality of a practical schedule with respect to an optimal schedule.

研究动机与目标

• 为同质、无通信开销模型中分块Cholesky分解的并行执行时间界限提供理论理解上的填补。
• 分析ALAP调度相较于ASAP调度在最小化完成时间方面的性能优势。
• 为任意处理单元数p推导出紧致的、闭式表达的执行时间下界。
• 通过与实际调度（包括ALAP和Kurzak等人方法）的对比，评估该下界的紧致程度。
• 为任务驱动线性代数算法的实际调度启发式方法的评估提供理论基础。

提出的方法

• 采用同质、以浮点运算权重为基准的任务图模型，无通信开销，聚焦于t×t分块矩阵的分块Cholesky分解。
• 应用ALAP调度启发式策略，从执行末尾开始反向调度任务，以实现更优的负载均衡并减少空闲时间。
• 通过任务依赖图的结构分析，推导出实现关键路径长度所需处理单元数的下界。
• 将任务集划分为两个子集，引入含√p的新型中间项表达式，以获得更紧致的执行时间下界。
• 在相同假设下，将理论边界与三种调度的模拟加速比进行比较：LAPACK（分叉-合并）、Kurzak等人方法及ALAP。
• 采用理论仿真框架，固定浮点运算权重（POTRF=1，TRSM=3，SYRK=3，GEMM=6），以确保比较的一致性。

实验结果

研究问题

• RQ1 在分块Cholesky分解中，实现关键路径长度所需的最少处理单元数是多少？
• RQ2 ALAP调度与ASAP调度在完成时间与处理器效率方面有何比较优势？
• RQ3 能否推导出一种改进朴素下界并反映任务图结构的执行时间闭式下界？
• RQ4 所推导的下界有多紧致？能否被实际调度启发式方法所匹配？
• RQ5 该理论框架能否推广至其他基于分块的线性代数算法（如LU或QR）？

主要发现

• 本文证明，任何调度都无法在少于0.185t²个处理单元的情况下实现关键路径长度为9t−10的目标，显著优于朴素下界0.11t²。
• 基于ALAP的调度策略可在少于0.25t² + 0.16t + 3个处理单元下完成于关键路径时间，相较于朴素下界0.50t²有显著改进。
• 推导出的执行时间下界为max(t³/p, t³/p − 3t²/p + 6√(2p) − 7, 9t−10)，其中中间项为关键创新贡献。
• 仿真结果表明，当t=40时，ALAP调度在p=343下达到关键路径，验证了其效率与理论边界的正确性。
• LAPACK风格的分叉-合并调度表现欠佳，归因于过度同步，凸显了ALAP调度的优势。
• 理论下界与最佳实际调度之间仍存在差距，表明在渐近分析中仍有进一步收紧的空间。