[Paper Review] Accelerating BLAS on Custom Architecture through Algorithm-Architecture Co-design

Farhad Merchant, Tarun Vatwani+4 more|arXiv (Cornell University)|Oct 20, 2016

Parallel Computing and Optimization Techniques23 references3 citations

TL;DR

This paper presents a co-design approach for accelerating Basic Linear Algebra Subprograms (BLAS) on a coarse-grained reconfigurable architecture (REDEFINE CGRA) by optimizing a custom Processing Element (PE) for Level-1, -2, and -3 BLAS operations. The PE achieves up to 74% of theoretical peak performance in double-precision GEMM, 40% in GEMV, and 20% in inner product, enabling 3–140× speedup over Intel, Nvidia GPUs, and FPGA platforms while delivering 35.7 Gflops/watt, demonstrating scalable performance through algorithm-architecture co-design.

ABSTRACT

Basic Linear Algebra Subprograms (BLAS) play key role in high performance and scientific computing applications. Experimentally, yesteryear multicore and General Purpose Graphics Processing Units (GPGPUs) are capable of achieving up to 15 to 57% of the theoretical peak performance at 65W to 240W respectively for compute bound operations like Double/Single Precision General Matrix Multiplication (XGEMM). For bandwidth bound operations like Single/Double precision Matrix-vector Multiplication (XGEMV) the performance is merely 5 to 7% of the theoretical peak performance in multicores and GPGPUs respectively. Achieving performance in BLAS requires moving away from conventional wisdom and evolving towards customized accelerator tailored for BLAS through algorithm-architecture co-design. In this paper, we present acceleration of Level-1 (vector operations), Level-2 (matrix-vector operations), and Level-3 (matrix-matrix operations) BLAS through algorithm architecture co-design on a Coarse-grained Reconfigurable Architecture (CGRA). We choose REDEFINE CGRA as a platform for our experiments since REDEFINE can be adapted to support domain of interest through tailor-made Custom Function Units (CFUs). For efficient sequential realization of BLAS, we present design of a Processing Element (PE) and perform micro-architectural enhancements in the PE to achieve up-to 74% of the theoretical peak performance of PE in DGEMM, 40% in DGEMV and 20% in double precision inner product (DDOT). We attach this PE to REDEFINE CGRA as a CFU and show the scalability of our solution. Finally, we show performance improvement of 3-140x in PE over commercially available Intel micro-architectures, ClearSpeed CSX700, FPGA, and Nvidia GPGPUs.

Motivation & Objective

• To address the performance saturation of legacy BLAS implementations on multicore and GPGPU platforms, which achieve only 15–57% of theoretical peak performance.
• To overcome the inefficiency in compute and memory resource utilization in conventional architectures for compute- and bandwidth-bound BLAS kernels.
• To design a domain-specific, energy-efficient Processing Element (PE) tailored for BLAS operations through micro-architectural enhancements.
• To integrate the optimized PE as a Custom Function Unit (CFU) in the REDEFINE CGRA and evaluate its scalability and performance across matrix sizes.

Proposed method

• Design a specialized Processing Element (PE) with micro-architectural enhancements to efficiently execute Level-1, -2, and -3 BLAS routines.
• Implement the PE on the REDEFINE Coarse-Grained Reconfigurable Architecture (CGRA), leveraging its ability to customize Custom Function Units (CFUs) for specific computational domains.
• Use Directed Acyclic Graph (DAG) analysis to model dataflow and identify parallelism in BLAS kernels, enabling efficient mapping to the PE.
• Partition large matrices into blocks (e.g., 10×10 or 20×20) and distribute computation across multiple PEs in a tiled array to exploit parallelism.
• Employ a tiled algorithmic approach to balance computation and communication, minimizing data movement overhead.
• Benchmark the PE and CGRA configuration against Intel, AMD, Nvidia GPUs, FPGA, and ClearSpeed CSX700 using CPI and Gflops/watt metrics for fair comparison.

Experimental results

Research questions

• RQ1 Can algorithm-architecture co-design break the performance saturation point observed in conventional multicore and GPGPU platforms for BLAS operations?
• RQ2 To what extent can micro-architectural enhancements in a custom PE improve performance and energy efficiency for BLAS kernels like DGEMM, DGEMV, and DDOT?
• RQ3 How does the scalability of the PE-based solution on REDEFINE CGRA vary with increasing matrix size and number of tiles?
• RQ4 What performance gains are achievable in terms of Gflops/watt and speedup over general-purpose and specialized accelerators for BLAS workloads?
• RQ5 Can a domain-specific PE achieve higher performance than off-the-shelf processors and GPUs while maintaining energy efficiency?

Key findings

• The custom PE achieves 74% of theoretical peak performance in double-precision GEMM (DGEMM), 40% in GEMV, and 20% in inner product (DDOT), significantly outperforming general-purpose processors.
• The PE delivers 35.7 Gflops/watt, a substantial improvement over Intel/AMD processors (0.02–0.25 Gflops/watt) and Nvidia GPUs.
• The PE achieves 3–140× speedup over Intel Core micro-architectures, 7–139× over Nvidia GPUs, 10× over Altera FPGA, and 3× over ClearSpeed CSX700 in terms of Gflops/watt.
• Parallel realization on REDEFINE CGRA with 2×2, 3×3, and 4×4 tile arrays achieves speedups of up to 4×, 9×, and 16× over the single-PE implementation, respectively, with speedup saturating at larger matrix sizes.
• The computation-to-communication ratio improves with matrix size (e.g., 10:1 for 20×20, 20:1 for 60×60), indicating reduced communication overhead in larger problems.
• The scalability of the solution is validated by consistent speedup gains proportional to the number of tiles used, confirming the effectiveness of the co-design approach.