[Paper Review] Towards a Multi-array Architecture for Accelerating Large-scale Matrix Multiplication on FPGAs

Junzhong Shen, Yuran Qiao+3 more|arXiv (Cornell University)|Mar 10, 2018

Parallel Computing and Optimization Techniques7 references3 citations

TL;DR

This paper proposes a scalable, highly configurable multi-array architecture for accelerating large-scale matrix multiplication on FPGAs by extending linear systolic arrays with dynamic configuration of processing elements (PEs) and PE arrays. It introduces a work-stealing load-balancing scheme and an analytical model to determine optimal design parameters, achieving up to 100.9 GFLOPS (98.6% of theoretical peak) on AlexNet layers with minimal resource utilization.

ABSTRACT

Large-scale floating-point matrix multiplication is a fundamental kernel in many scientific and engineering applications. Most existing work only focus on accelerating matrix multiplication on FPGA by adopting a linear systolic array. This paper towards the extension of this architecture by proposing a scalable and highly configurable multi-array architecture. In addition, we propose a work-stealing scheme to ensure the equality in the workload partition among multiple linear arrays. Furthermore, an analytical model is developed to determine the optimal design parameters. Experiments on a real-life convolutional neural network (CNN) show that we can obtain the optimal extension of the linear array architecture.

Motivation & Objective

• To address the limitations of existing linear systolic array architectures in scaling for large-scale matrix multiplication on FPGAs.
• To enable dynamic configuration of the number of PEs and parallel PE arrays to adapt to varying problem sizes.
• To ensure workload balance across multiple PE arrays through a work-stealing scheme.
• To develop an analytical model that determines optimal design parameters (block size and number of arrays) for performance and memory efficiency.
• To validate the architecture's performance on real-world deep learning workloads like AlexNet.

Proposed method

• The proposed architecture uses multiple linear PE arrays connected via configurable multiplexers, enabling independent or cooperative operation modes.
• A work-stealing scheme dynamically balances workloads across PE arrays by redistributing tasks when imbalances occur.
• The Matrices Processing Engine (MPE) employs a fully pipelined, floating-point multiply-and-accumulate unit per PE with dual input registers and three FIFOs for data flow.
• An analytical model estimates memory bandwidth and data traffic to guide optimal configuration of block size $S_i$ and number of arrays $N_p$.
• The model uses Equation 9 to reduce the design space by fixing total PEs ($P_m \times P$) and minimizing the range of total execution time $T_{‘total}$.
• The system is implemented on a Xilinx VC709 FPGA with DDR3 memory, using Vivado 2016.4 for synthesis and timing analysis.

Experimental results

Research questions

• RQ1 How can a linear systolic array architecture be effectively extended to scale for large-scale matrix multiplication on FPGAs?
• RQ2 What design parameters (block size $S_i$, number of PE arrays $N_p$) maximize performance and memory efficiency?
• RQ3 How can workload imbalance across multiple PE arrays be mitigated during computation?
• RQ4 To what extent can the analytical model predict actual execution time and guide optimal configuration?
• RQ5 Does the multi-array architecture outperform single-array or fully expanded array designs in real-world workloads?

Key findings

• The optimal configuration for AlexNet's fully connected layer (fc-6) achieved 100.9 GFLOPS, representing 98.6% of the theoretical peak performance.
• The case $(N_p=1, S_i=32)$ outperformed $(N_p=2, S_i=16)$ despite lower PE count, due to higher effective memory bandwidth (1.6 GB/s vs. 1.4 GB/s).
• For all AlexNet layers, the proposed architecture with optimal $<N_p, S_i>$ outperformed both single-array ($N_p=1$) and fully expanded array ($N_p=4$) configurations.
• The analytical model's predicted lower bound closely matched actual execution times when memory bandwidth was sufficient, but diverged when memory-bound.
• Resource utilization remained below 50% (e.g., 44.44% LUTs), enabling a high clock frequency of 200 MHz and efficient performance scaling.
• The work-stealing scheme effectively balanced workloads across PE arrays, preventing idle cycles and improving overall utilization.