[Paper Review] Tiled QR factorization algorithms

Henricus Bouwmeester, Mathias Jacquelin+2 more|arXiv (Cornell University)|Apr 22, 2011

Interconnection Networks and Systems12 references4 citations

TL;DR

This paper introduces tiled QR factorization algorithms using Fibonacci and Greedy scheduling strategies for rectangular matrices, optimizing parallelism in multicore architectures. It proves both algorithms are asymptotically optimal for matrices where p = q²f(q) with lim f(q) = 0, and experiments show superior performance over existing methods like PlasmaTree and FlatTree, especially for tall matrices.

ABSTRACT

This work revisits existing algorithms for the QR factorization of rectangular matrices composed of p-by-q tiles, where p >= q. Within this framework, we study the critical paths and performance of algorithms such as Sameh and Kuck, Modi and Clarke, Greedy, and those found within PLASMA. Although neither Modi and Clarke nor Greedy is optimal, both are shown to be asymptotically optimal for all matrices of size p = q^2 f(q), where f is any function such that \lim_{+\infty} f= 0. This novel and important complexity result applies to all matrices where p and q are proportional, p = λq, with λ>= 1, thereby encompassing many important situations in practice (least squares). We provide an extensive set of experiments that show the superiority of the new algorithms for tall matrices.

Motivation & Objective

• Address the need for highly parallel QR factorization algorithms in multicore architectures to accelerate numerical linear algebra workloads.
• Improve upon existing tiling strategies (e.g., Sameh-Kuck, FlatTree, PlasmaTree) by introducing new scheduling algorithms that enhance parallelism without tuning parameters.
• Establish theoretical optimality of Fibonacci and Greedy algorithms in the asymptotic regime for tall and skinny matrices.
• Demonstrate through extensive experiments that the new algorithms outperform state-of-the-art implementations in terms of critical path length and execution time.

Proposed method

• Adapt Fibonacci and Greedy scheduling strategies to tile-based QR factorization, replacing scalar operations with tile-level transformations.
• Model the computation as a dependency graph where tiles represent computational units and edges represent data dependencies.
• Use a critical path analysis to evaluate the theoretical performance bounds of each algorithm, focusing on parallelism and latency.
• Implement and benchmark the new algorithms using TT (tall tiles) and TS (square tiles) kernels on multicore systems with double and complex double precision.
• Compare results against established algorithms like FlatTree, PlasmaTree, and Sameh-Kuck using metrics such as critical path length and execution time.
• Leverage compact WY representations and Level 3 BLAS for efficient tile updates, combining the stability of Householder reflections with the parallelism of Givens-style scheduling.

Experimental results

Research questions

• RQ1 Can Fibonacci and Greedy scheduling strategies be effectively adapted to tile-based QR factorization to improve parallelism?
• RQ2 Under what matrix size conditions are Fibonacci and Greedy algorithms asymptotically optimal?
• RQ3 How do the new algorithms compare in practice to existing implementations like PlasmaTree and FlatTree in terms of execution time and critical path length?
• RQ4 Does the absence of tuning parameters (e.g., domain size) in Fibonacci and Greedy lead to more robust performance across diverse matrix shapes?
• RQ5 To what extent do tile-based algorithms with TT kernels improve performance compared to square tiles in multicore settings?

Key findings

• Fibonacci and Greedy algorithms are asymptotically optimal for all matrices with p = q²f(q) where lim f(q) = 0, which includes the important case of p = λq with λ ≥ 1.
• For tall matrices (e.g., p = 40, q = 1), the Greedy algorithm achieves a 28.09% gain over PlasmaTree (TT) in execution time, with overhead relative to Greedy at 1.0153 for PlasmaTree (TS) and 1.0152 for PlasmaTree (TT).
• The Fibonacci algorithm reduces execution time by up to 28.15% compared to Greedy for double complex arithmetic in the case p=40, q=1, with an overhead of 0.7185 relative to Greedy.
• In double precision, the Greedy algorithm achieves a 13.64% performance gain over PlasmaTree (TT) for p=40, q=2, with an overhead of 0.8636.
• The new algorithms outperform PlasmaTree (TT) across all tested matrix sizes, with gains ranging from 0.0067 to 0.1364 in execution time relative to Greedy.
• The experimental results confirm that the new algorithms are robust and do not require tuning parameters such as domain size, unlike prior approaches like [12].