[Paper Review] BLISlab: A Sandbox for Optimizing GEMM

Jianyu Huang, Robert A. Geijn|arXiv (Cornell University)|Sep 1, 2016

Parallel Computing and Optimization Techniques10 references9 citations

TL;DR

BLISlab is a pedagogical sandbox built on the BLIS framework to teach high-performance optimization of matrix-matrix multiplication (GEMM) on modern CPUs. It guides users through blocking, micro-kernel implementation using AVX/AVX2 intrinsics or assembly, and loop-level parallelization, with the goal of achieving high performance on Intel Haswell and other architectures through systematic code tuning and micro-kernel optimization.

ABSTRACT

Matrix-matrix multiplication is a fundamental operation of great importance to scientific computing and, increasingly, machine learning. It is a simple enough concept to be introduced in a typical high school algebra course yet in practice important enough that its implementation on computers continues to be an active research topic. This note describes a set of exercises that use this operation to illustrate how high performance can be attained on modern CPUs with hierarchical memories (multiple caches). It does so by building on the insights that underly the BLAS-like Library Instantiation Software (BLIS) framework by exposing a simplified "sandbox" that mimics the implementation in BLIS. As such, it also becomes a vehicle for the "crowd sourcing" of the optimization of BLIS. We call this set of exercises BLISlab.

Motivation & Objective

• To provide an educational platform for teaching high-performance GEMM optimization on modern CPU architectures with hierarchical memory.
• To enable researchers and students to experiment with blocking parameters and micro-kernel implementations to achieve high performance.
• To crowdsource the development of optimal micro-kernels for the BLIS framework across diverse CPU architectures.
• To demonstrate how algorithmic insights and low-level optimizations (e.g., vectorization, loop tiling) can be systematically applied to achieve high performance in GEMM.
• To serve as a training ground for understanding the performance-critical aspects of BLAS-level kernels, especially for level-3 operations.

Proposed method

• Uses a simplified BLIS-like framework to abstract away low-level complexity and focus on performance-critical components.
• Employs loop tiling (blocking) with configurable parameters m_C, n_C, k_C to optimize data locality across cache levels.
• Provides a reference micro-kernel in naive C, with templates for optimized versions using Intel AVX/AVX2 intrinsics or hand-written assembly.
• Supports micro-kernel implementation via two main patterns: broadcast-based and butterfly permutation-based 4×4 rank-1 updates.
• Enables performance measurement and tuning through configurable function pointers and compilation settings.
• Introduces OpenMP parallelization at the outermost loop level to exploit multi-core execution, with guidance on loop selection for optimal scaling.

Experimental results

Research questions

• RQ1 How can a systematic, pedagogical approach be used to teach high-performance GEMM optimization on modern CPUs?
• RQ2 What are the key performance bottlenecks in naive GEMM implementations, and how can they be addressed through tiling and micro-kernel optimization?
• RQ3 How do vector intrinsics (AVX/AVX2) and hand-optimized assembly improve GEMM performance compared to high-level C code?
• RQ4 What are the performance trade-offs of parallelizing different levels of the GEMM computation loop nest?
• RQ5 Can a community-driven, open-source approach effectively identify and contribute the most performant micro-kernels for the BLIS framework?

Key findings

• The BLISlab framework successfully enables users to achieve significant performance improvements over naive C implementations through strategic loop tiling and micro-kernel optimization.
• Micro-kernels implemented using AVX/AVX2 intrinsics or hand-written assembly achieve substantially higher performance than the baseline C version, particularly on Intel Haswell architecture.
• The 4×4 rank-1 update pattern using FMA instructions and vectorized broadcasting or shuffling operations is a key performance primitive in optimized micro-kernels.
• Proper choice of blocking parameters (m_C, n_C, k_C) significantly improves data locality and reduces memory bandwidth pressure.
• OpenMP parallelization of the outermost loop in the GEMM kernel can effectively exploit multi-core processors, with performance scaling observed on multi-threaded systems.
• The framework successfully demonstrates that high-performance GEMM can be achieved through a combination of algorithmic insight, compiler intrinsics, and low-level assembly, even in a teaching context.