[Paper Review] A Metaprogramming and Autotuning Framework for Deploying Deep Learning Applications

Matthew W. Moskewicz, Ali Jannesari+1 more|arXiv (Cornell University)|Nov 21, 2016

Advanced Neural Network Applications18 references3 citations

TL;DR

This paper presents Boda, a metaprogramming and autotuning framework that enables portable, high-performance GPU code generation for deep learning operations across diverse hardware platforms. By combining C++-based metaprogramming with automated tuning, the framework achieves performance competitive with vendor libraries on NVIDIA GPUs and enables efficient porting to Qualcomm and AMD GPUs with minimal manual effort.

ABSTRACT

In recent years, deep neural networks (DNNs), have yielded strong results on a wide range of applications. Graphics Processing Units (GPUs) have been one key enabling factor leading to the current popularity of DNNs. However, despite increasing hardware flexibility and software programming toolchain maturity, high efficiency GPU programming remains difficult: it suffers from high complexity, low productivity, and low portability. GPU vendors such as NVIDIA have spent enormous effort to write special-purpose DNN libraries. However, on other hardware targets, especially mobile GPUs, such vendor libraries are not generally available. Thus, the development of portable, open, high-performance, energy-efficient GPU code for DNN operations would enable broader deployment of DNN-based algorithms. Toward this end, this work presents a framework to enable productive, high-efficiency GPU programming for DNN computations across hardware platforms and programming models. In particular, the framework provides specific support for metaprogramming, autotuning, and DNN-tailored data types. Using our framework, we explore implementing DNN operations on three different hardware targets: NVIDIA, AMD, and Qualcomm GPUs. On NVIDIA GPUs, we show both portability between OpenCL and CUDA as well competitive performance compared to the vendor library. On Qualcomm GPUs, we show that our framework enables productive development of target-specific optimizations, and achieves reasonable absolute performance. Finally, On AMD GPUs, we show initial results that indicate our framework can yield reasonable performance on a new platform with minimal effort.

Motivation & Objective

• To address the high complexity and low portability of hand-optimized GPU kernels for deep learning operations.
• To reduce the engineering effort required to deploy high-performance DNN kernels on new or non-vendor-supported hardware platforms.
• To enable performance portability across programming models (CUDA and OpenCL) and hardware targets (NVIDIA, AMD, Qualcomm GPUs).
• To provide a productive, extensible framework that supports metaprogramming and autotuning for DNN-specific kernels.
• To demonstrate that automated tuning and metaprogramming can yield competitive performance on mobile and emerging GPU platforms without vendor-specific libraries.

Proposed method

• The framework uses a C++-hosted string-template-based metaprogramming approach to generate optimized C-level GPU kernels from high-level DNN computation graphs.
• It abstracts low-level differences between CUDA and OpenCL, enabling syntactic and semantic compatibility across programming models.
• The framework supports static loop unrolling and code generation via metacode constructs that mirror actual kernel code.
• It integrates autotuning to explore and select optimal kernel configurations (e.g., tile sizes, data types) across hardware platforms.
• It enables profiling and analysis of generated code for debugging and optimization guidance.
• The framework reuses tuned kernels from one platform (e.g., NVIDIA) as a starting point for tuning on new platforms (e.g., Qualcomm, AMD).

Experimental results

Research questions

• RQ1 Can a metaprogramming framework achieve performance competitive with vendor-optimized libraries like cuDNN on NVIDIA GPUs?
• RQ2 To what extent can the same high-level DNN kernel specification be ported across different GPU programming models (CUDA vs. OpenCL) with minimal changes?
• RQ3 How effective is autotuning in enabling rapid performance portability to new hardware platforms such as mobile GPUs (Qualcomm) and emerging architectures (AMD)?
• RQ4 Can the framework reduce the development effort required to generate efficient GPU kernels for non-vendor-supported hardware?
• RQ5 What is the role of metaprogramming in simplifying the generation and debugging of highly optimized DNN kernels?

Key findings

• On NVIDIA GPUs, the framework achieves performance competitive with cuDNN using both CUDA and OpenCL, demonstrating effective performance and programming model portability.
• On Qualcomm mobile GPUs, the framework enabled a developer with no prior knowledge of the target to achieve reasonable performance in just a few weeks of part-time effort.
• On AMD GPUs, autotuning of pre-existing code produced a functional and reasonably performant baseline with minimal manual tuning, validating the framework’s role as a foundation for platform-specific optimization.
• The metaprogramming approach using C++ templates for code generation proved more maintainable and debuggable than lower-level alternatives like Perl-generated assembly.
• The framework’s ability to reuse tuned configurations across platforms significantly reduces the total engineering cost of porting DNN kernels to new hardware.
• The results suggest that the framework can serve as a viable, open, and vendor-neutral alternative to proprietary DNN libraries for high-performance inference.