[Paper Review] High-Performance GPU-to-CPU Transpilation and Optimization via High-Level Parallel Constructs

William S. Moses, Ivan R. Ivanov+4 more|arXiv (Cornell University)|Jul 1, 2022

Parallel Computing and Optimization Techniques4 citations

TL;DR

This paper presents a compiler framework that automatically transpiles CUDA kernels to high-performance CPU code using high-level parallel constructs in MLIR, enabling efficient optimization and execution on CPUs without manual porting. It achieves a 76% geomean speedup over handwritten OpenMP code on the Rodinia benchmark suite and a 2.7× speedup over PyTorch's native CPU backend by leveraging transpiled CUDA kernels on Fugaku.

ABSTRACT

While parallelism remains the main source of performance, architectural implementations and programming models change with each new hardware generation, often leading to costly application re-engineering. Most tools for performance portability require manual and costly application porting to yet another programming model. We propose an alternative approach that automatically translates programs written in one programming model (CUDA), into another (CPU threads) based on Polygeist/MLIR. Our approach includes a representation of parallel constructs that allows conventional compiler transformations to apply transparently and without modification and enables parallelism-specific optimizations. We evaluate our framework by transpiling and optimizing the CUDA Rodinia benchmark suite for a multi-core CPU and achieve a 76% geomean speedup over handwritten OpenMP code. Further, we show how CUDA kernels from PyTorch can efficiently run and scale on the CPU-only Supercomputer Fugaku without user intervention. Our PyTorch compatibility layer making use of transpiled CUDA PyTorch kernels outperforms the PyTorch CPU native backend by 2.7$ imes$.

Motivation & Objective

• To address the performance portability challenge in HPC and machine learning, where GPU-optimized code must be re-implemented for CPU-only systems like Fugaku.
• To overcome the limitations of existing tools that require manual re-implementation by enabling fully automated, high-level transpilation from CUDA to CPU threads.
• To enable conventional compiler optimizations—such as loop-invariant code motion and common subexpression elimination—on GPU-style parallel constructs by modeling synchronization via memory semantics.
• To demonstrate efficient execution of PyTorch CUDA kernels on CPU-only supercomputers without GPU hardware, using a PyTorch-compatible transpilation layer.

Proposed method

• The framework uses MLIR's nested-module structure to represent multi-level GPU parallelism, including block-level synchronization and shared memory, preserving high-level program structure.
• It models GPU barriers as memory semantics operations, enabling integration with standard compiler optimizations instead of treating them as opaque barriers.
• The system extends Polygeist's C/C++ frontend to parse CUDA code and emit MLIR with preserved parallel constructs and data layout.
• It applies standard compiler optimizations (e.g., loop-invariant code motion) transparently to GPU-style parallel regions by analyzing memory behavior.
• The transpiled code is compiled to OpenMP-parallel CPU code, enabling efficient multi-core execution on commodity and exascale CPUs.
• A PyTorch compatibility layer transpiles CUDA kernels at runtime, allowing GPU-optimized models to run on CPU-only systems like Fugaku.

Experimental results

Research questions

• RQ1 Can a compiler automatically transpile CUDA kernels to high-performance CPU code while preserving and optimizing parallelism?
• RQ2 Can high-level parallel constructs be represented in a way that enables standard compiler optimizations without modification?
• RQ3 Can GPU-style synchronization (e.g., block-wide barriers) be modeled via memory semantics to allow integration with existing optimizations?
• RQ4 Can transpiled CUDA code outperform handwritten OpenMP implementations on CPU architectures?
• RQ5 Can PyTorch CUDA kernels be efficiently executed on CPU-only supercomputers like Fugaku without GPU hardware?

Key findings

• The transpiled CUDA code achieves a 76% geomean speedup over handwritten OpenMP implementations on the Rodinia benchmark suite when executed on a commodity CPU.
• The framework enables efficient execution of PyTorch CUDA kernels on the CPU-only Fugaku supercomputer, outperforming the native PyTorch CPU backend by 2.7× on average.
• The system maintains the original kernel schedule on CPU, with innermost loop serialization improving performance due to reduced thread divergence and better memory access patterns.
• The use of memory-semantic barrier modeling allows standard compiler optimizations—such as loop-invariant code motion and common subexpression elimination—to be applied directly to GPU-style parallel regions.
• The framework successfully transpiles and optimizes complex kernels from PyTorch, including ResNet-50, demonstrating compatibility with real-world deep learning workloads.
• The approach enables performance portability without manual re-implementation, reducing the cost of adapting GPU-optimized code to CPU-only or heterogeneous HPC systems.