[Paper Review] Resource-Aware Just-in-Time OpenCL Compiler for Coarse-Grained FPGA Overlays

Abhishek Kumar Jain, Douglas L. Maskell+1 more|arXiv (Cornell University)|May 8, 2017

Parallel Computing and Optimization Techniques17 references3 citations

TL;DR

This paper presents a resource-aware just-in-time (JIT) OpenCL compiler targeting a DSP-based coarse-grained FPGA overlay, enabling sub-second compilation times by bypassing traditional FPGA place-and-route. The approach achieves a 1,250× speedup in PAR time compared to Vivado on a workstation and a 300× speedup when running on the Zynq ARM processor, while supporting runtime kernel replication and configuration in under 50 μs.

ABSTRACT

FPGA vendors have recently started focusing on OpenCL for FPGAs because of its ability to leverage the parallelism inherent to heterogeneous computing platforms. OpenCL allows programs running on a host computer to launch accelerator kernels which can be compiled at run-time for a specific architecture, thus enabling portability. However, the prohibitive compilation times (specifically the FPGA place and route times) are a major stumbling block when using OpenCL tools from FPGA vendors. The long compilation times mean that the tools cannot effectively use just-in-time (JIT) compilation or runtime performance scaling. Coarse-grained overlays represent a possible solution by virtue of their coarse granularity and fast compilation. In this paper, we present a methodology for run-time compilation of OpenCL kernels to a DSP block based coarse-grained overlay, rather than directly to the fine-grained FPGA fabric. The proposed methodology allows JIT compilation and on-demand resource-aware kernel replication to better utilize available overlay resources, raising the abstraction level while reducing compile times significantly. We further demonstrate that this approach can even be used for run-time compilation of OpenCL kernels on the ARM processor of the embedded heterogeneous Zynq device.

Motivation & Objective

• Address the prohibitively long compilation times of FPGA OpenCL tools, which hinder JIT compilation and runtime performance scaling.
• Overcome the limitations of fine-grained FPGA fabrics by using a coarse-grained overlay architecture to accelerate place-and-route.
• Enable on-demand, resource-aware kernel replication at runtime to improve hardware utilization and performance.
• Demonstrate end-to-end JIT compilation on embedded processors, such as the Zynq ARM, to support portable and dynamic acceleration.
• Quantify the trade-off between compilation speed and resource overhead in overlay-based FPGA compilation.

Proposed method

• Design a custom automated mapping flow using the LLVM front-end to compile OpenCL kernels to a DSP block-based coarse-grained overlay.
• Implement a vendor-independent compilation pipeline that abstracts away low-level FPGA details, reducing compilation complexity.
• Use a modified place-and-route (PAR) tool flow tailored for the overlay architecture to drastically reduce compilation time.
• Deploy the OpenCL runtime infrastructure (pocl) on the Zynq’s ARM processor to enable JIT compilation and kernel execution on the embedded system.
• Support dynamic kernel replication based on available overlay resources, enabling runtime performance scaling.
• Measure and compare PAR times, frequency, resource utilization (DSPs, logic slices), and configuration time between overlay and native FPGA implementations.

Experimental results

Research questions

• RQ1 Can a coarse-grained FPGA overlay significantly reduce OpenCL kernel compilation time compared to traditional FPGA place-and-route?
• RQ2 To what extent can JIT compilation be enabled on embedded FPGA platforms like Zynq using an overlay-based approach?
• RQ3 What is the performance and resource overhead of using a coarse-grained overlay compared to direct FPGA implementation?
• RQ4 Can on-demand, resource-aware kernel replication be effectively supported in a JIT environment using this overlay methodology?
• RQ5 How does the configuration time of the overlay compare to that of a full FPGA fabric?

Key findings

• The overlay-based place-and-route process is approximately 1,250× faster than Vivado’s native FPGA flow on a high-end workstation.
• When running on the Zynq’s ARM processor, the overlay PAR process is still over 300× faster than the native FPGA flow.
• The overlay incurs a 3.4× increase in DSP block usage and a 32× increase in logic slice utilization compared to direct FPGA implementation.
• Maximum frequency improves by 1.6× on the overlay due to simplified routing and predictable placement.
• Configuration time is reduced by 750×, from 31.6 ms on the FPGA to 42.4 μs on the 8×8 overlay.
• The system successfully supports JIT compilation and runtime kernel replication using only the Zynq’s ARM processor, demonstrating feasibility on embedded platforms.