[Paper Review] Cross-Platform Performance Portability Using Highly Parametrized SYCL Kernels

John Lawson, Mehdi Goli+3 more|arXiv (Cornell University)|Apr 10, 2019

Parallel Computing and Optimization Techniques17 references4 citations

TL;DR

This paper demonstrates that highly parameterized SYCL kernels achieve performance competitive with hand-optimized vendor libraries (e.g., cuBLAS, ARM Compute Library, MKL-DNN) across diverse architectures like ARM HiKey 960 and Intel i7-6700K. By leveraging C++ templates and SYCL's portability, the authors instantiate kernels tuned per hardware, enabling cross-platform performance portability without sacrificing efficiency.

ABSTRACT

Over recent years heterogeneous systems have become more prevalent across HPC systems, with over 100 supercomputers in the TOP500 incorporating GPUs or other accelerators. These hardware platforms have different performance characteristics and optimization requirements. In order to make the most of multiple accelerators a developer has to provide implementations of their algorithms tuned for each device. Hardware vendors provide libraries targeting their devices specifically, which provide good performance but frequently have different API designs, hampering portability. The SYCL programming model allows users to write heterogeneous programs using completely standard C++, and so developers have access to the power of C++ templates when developing compute kernels. In this paper we show that by writing highly parameterized kernels for matrix multiplies and convolutions we achieve performance competitive with vendor implementations across different architectures. Furthermore, tuning for new devices amounts to choosing the combinations of kernel parameters that perform best on the hardware.

Motivation & Objective

• To address the challenge of performance portability across heterogeneous accelerators with differing optimization needs.
• To reduce reliance on vendor-specific libraries that hinder portability due to divergent APIs and hardware targeting.
• To evaluate whether highly parameterized SYCL kernels can match or exceed the performance of hand-optimized libraries like cuBLAS, ARM Compute Library, and MKL-DNN.
• To demonstrate that tuning kernel parameters per architecture enables efficient, portable performance across diverse hardware platforms.

Proposed method

• The authors implement highly parameterized SYCL kernels for matrix multiplication and convolution operations using C++ templates to express kernel configurations.
• They instantiate these kernels with specific template parameters (e.g., block sizes, tile dimensions, data types) tailored to each target device’s architecture.
• Performance is evaluated using standard deep learning benchmarks: VGG-16 and ResNet-50 convolution layers on ARM HiKey 960 and Intel i7-6700K systems.
• The SYCL-based implementation (SYCL-DNN) is compared against vendor-optimized libraries: ARM Compute Library (CPU/NEON and GPU/OpenCL), MKL-DNN, and cuBLAS.
• Benchmarking is conducted with varying batch sizes to assess real-world performance under typical workloads.
• The approach relies on SYCL’s ability to generate optimized, portable kernels via compile-time template specialization while maintaining standard C++ syntax.

Experimental results

Research questions

• RQ1 Can highly parameterized SYCL kernels achieve performance competitive with hand-optimized, vendor-specific libraries across diverse hardware platforms?
• RQ2 To what extent can kernel tuning via template parameters replace the need for architecture-specific hand-optimization?
• RQ3 How does the performance of SYCL-DNN compare to ARM Compute Library and MKL-DNN on ARM and Intel platforms, respectively?
• RQ4 Can a single, portable kernel framework deliver high performance across CPUs, GPUs, and accelerators without sacrificing portability?

Key findings

• On the ARM HiKey 960, SYCL-DNN achieved between 90 and 150 gigaflops across VGG layers, outperforming the NEON CPU implementation and matching or exceeding the OpenCL GPU version in most cases.
• On the Intel i7-6700K, SYCL-DNN on GPU consistently outperformed MKL-DNN on CPU for VGG-16 convolutions, achieving up to 420 gigaflops.
• For ResNet-50, MKL-DNN on CPU was faster than SYCL-DNN on both CPU and GPU, with peak performance of 366 gigaflops versus 244 gigaflops.
• In VGG-16 benchmarks on ARM, the ARM Compute Library OpenCL kernel outperformed SYCL-DNN in 3×3 convolutions, but SYCL-DNN remained competitive overall.
• The results confirm that SYCL kernels with appropriate template parameterization can achieve performance within 10–20% of vendor-optimized libraries on both ARM and Intel platforms.
• The study demonstrates that tuning kernel parameters per device enables effective performance portability without sacrificing efficiency, even across architectures with significantly different performance characteristics.