[Paper Review] A flexible FPGA accelerator for convolutional neural networks

Kingshuk Majumder, Nema, Shubham+1 more|arXiv (Cornell University)|Dec 16, 2019

CCD and CMOS Imaging Sensors31 references4 citations

TL;DR

This paper presents a flexible, FPGA-based CNN accelerator using 1D systolic arrays of processing elements that dynamically adapts to varying CNN layer shapes without reconfiguration, minimizing underutilization and off-chip memory access through comprehensive data reuse. The design achieves high performance—sustaining a significant fraction of theoretical peak—on a real Xilinx VC709 FPGA, with full TensorFlow integration via a codesigned software stack.

ABSTRACT

Though CNNs are highly parallel workloads, in the absence of efficient on-chip memory reuse techniques, an accelerator for them quickly becomes memory bound. In this paper, we propose a CNN accelerator design for inference that is able to exploit all forms of reuse available to minimize off-chip memory access while increasing utilization of available resources. The proposed design is composed of cores, each of which contains a one-dimensional array of processing elements. These cores can exploit different types of reuse available in CNN layers of varying shapes without requiring any reconfiguration; in particular, our design minimizes underutilization due to problem sizes that are not perfect multiples of the underlying hardware array dimensions. A major obstacle in the adoption of FPGAs as a platform for CNN inference is the difficulty to program these devices using hardware description languages. Our end goal is to also address this, and we develop preliminary software support via a codesign in order to leverage the accelerator through TensorFlow, a dominant high-level programming model. Our framework takes care of tiling and scheduling of neural network layers and generates necessary low-level commands to execute the CNN. Experimental evaluation on a real system with a PCI-express based Xilinx VC709 board demonstrates the effectiveness of our approach. As a result of an effective interconnection, the design maintains a high frequency when we scale the number of PEs. The sustained performance overall is a good fraction of the accelerator's theoretical peak performance.

Motivation & Objective

• Address the challenge of memory-bound performance in FPGA-based CNN accelerators by maximizing on-chip data reuse across multiple dimensions.
• Overcome the limitations of fixed 2D PE arrays by using flexible 1D systolic cores that adapt to arbitrary layer shapes and avoid reconfiguration.
• Minimize underutilization due to non-perfect multiples of array dimensions by enabling dynamic tiling and mapping of CNN layer tiles.
• Enable seamless integration with high-level frameworks like TensorFlow through a codesigned software stack, abstracting low-level FPGA complexity.
• Achieve high clock frequency and performance scalability across varying numbers of processing elements on real FPGA hardware.

Proposed method

• Design a core-level architecture based on a 1D array of processing elements (PEs) that supports convolution operations for arbitrary tile shapes of CNN layers.
• Implement a custom interconnect and dataflow mechanism to exploit input feature map reuse, weight reuse, partial sum reuse, and convolutional reuse across input, output, and channel dimensions.
• Use tiling and scheduling heuristics within a codesigned software framework to map layers to the accelerator while minimizing off-chip memory bandwidth.
• Develop a TensorFlow XLA backend and device registration to enable end-to-end execution of CNN models on the FPGA without low-level hardware coding.
• Leverage high-level synthesis and hardware-aware scheduling to maintain high clock frequency even at near-maximal PE utilization.
• Generate low-level commands from the high-level graph representation to drive the FPGA accelerator at runtime.

Experimental results

Research questions

• RQ1 How can an FPGA-based CNN accelerator efficiently exploit all forms of data reuse (input, weights, partial sums) across diverse CNN layer shapes without reconfiguration?
• RQ2 To what extent can a 1D systolic array architecture minimize underutilization caused by irregular layer dimensions compared to 2D arrays?
• RQ3 Can a flexible, reconfigurable FPGA accelerator sustain high performance and frequency while scaling the number of PEs?
• RQ4 How effectively can a high-level deep learning framework like TensorFlow be integrated with a low-level FPGA accelerator through a codesigned software stack?
• RQ5 What is the achievable performance relative to theoretical peak, and how does it compare to prior open-source FPGA CNN accelerators?

Key findings

• The 1D systolic array design sustains a high fraction of the theoretical peak performance—significantly higher than previously published open FPGA CNN accelerators with similar hardware setups.
• The design maintains high clock frequency even when scaling the number of PEs, indicating effective resource utilization and minimal area overhead from interconnect complexity.
• The software stack enables transparent execution of CNN models via TensorFlow, with automatic tiling and scheduling handled by the codesign flow.
• The accelerator achieves 652 DSP slices usage and higher performance than prior works like Zhang et al. [36] (61.2 GFLOPs) and Caffeine [35] (488 GOPs), despite using lower-precision int8 operations.
• The approach minimizes underutilization due to non-multiples of array dimensions by dynamically mapping tiles to 1D cores, enabling efficient use of all PEs across diverse layer shapes.
• Experimental results on a PCI-express-based Xilinx VC709 FPGA confirm the effectiveness of the design in reducing off-chip memory access and sustaining high throughput.