[Paper Review] TinyCNN: A Tiny Modular CNN Accelerator for Embedded FPGA

Ali Jahanshahi|arXiv (Cornell University)|Nov 15, 2019

CCD and CMOS Imaging Sensors16 references4 citations

TL;DR

This paper proposes TinyCNN, a modular framework for designing and deploying tiny, resource-aware CNN accelerators on embedded FPGAs. It enables hardware-aware CNN design via a software backend, automatic RTL generation using CHISEL, and precision tuning via simulation to minimize accuracy loss—achieving up to 15.75× speedup with only 3% accuracy degradation when using 16-bit fixed-point arithmetic compared to 32-bit floating-point on ARM.

ABSTRACT

In recent years, Convolutional Neural Network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are computational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited on-chip memory size limit the performance of FPGA accelerator for CNN. In this paper, we propose a framework for designing CNN accelerator on embedded FPGA for image classification. The proposed framework provides a tool for FPGA resource-aware design space exploration of CNNs and automatically generates the hardware description of the CNN to be programmed on a target FPGA. The framework consists of three main backends; software, hardware generation, and simulation/precision adjustment. The software backend serves as an API to the designer to design the CNN and train it according to the hardware resources that are available. Using the CNN model, hardware backend generates the necessary hardware components and integrates them to generate the hardware description of the CNN. Finaly, Simulation/precision adjustment backend adjusts the inter-layer precision units to minimize the classification error. We used 16-bit fixed-point data in a CNN accelerator (FPGA) and compared it to the exactly similar software version running on an ARM processor (32-bit floating point data). We encounter about 3% accuracy loss in classification of the accelerated (FPGA) version. In return, we got up to 15.75x speedup by classifying with the accelerated version on the FPGA.

Motivation & Objective

• To address the challenge of deploying computationally intensive CNNs on resource-constrained embedded FPGAs.
• To reduce the design complexity and time-to-deployment of custom CNN accelerators on FPGAs through automation.
• To minimize accuracy loss when using low-precision fixed-point arithmetic (16-bit) in CNN inference on FPGAs.
• To provide a general-purpose, modular framework for CNN accelerator generation that supports hardware resource constraints.
• To enable efficient hardware-software co-design by integrating CNN training, RTL generation, and precision tuning in a unified pipeline.

Proposed method

• The software backend uses Python to design and train CNNs, with real-time hardware resource estimation to guide architecture choices.
• The hardware backend generates RTL code using CHISEL, automatically composing convolutional, pooling, and fully connected layers based on the CNN architecture.
• The simulation and precision adjustment backend iteratively tunes the bit-width of inter-layer data paths (integer and fractional parts) to minimize classification error.
• The framework supports two hardware modes: shared (one convolution unit for all layers) and exclusive (dedicated unit per layer), with the latter reducing access contention.
• The entire pipeline is integrated using Xilinx SDSoC 2016.4, enabling HLS-based synthesis and programming of the Zynq SoC FPGA.
• The framework uses CIFAR-10 with grayscale inputs and a small 5-layer CNN to validate performance on a PYNQ Zynq-7000 board.

Experimental results

Research questions

• RQ1 Can a fully automated framework reduce the design effort and time-to-deployment for CNN accelerators on embedded FPGAs?
• RQ2 To what extent can 16-bit fixed-point arithmetic preserve classification accuracy compared to 32-bit floating-point in FPGA-based CNN inference?
• RQ3 How does hardware resource utilization, particularly BRAM, constrain CNN design on small FPGAs, and can the framework mitigate this?
• RQ4 Does exclusive convolution unit allocation improve performance over shared allocation in embedded FPGA accelerators?
• RQ5 Can iterative precision tuning of data path bit-width significantly reduce accuracy loss in low-precision CNNs?

Key findings

• The TinyCNN framework achieved a 15.75× speedup over the ARM Cortex-A9 software implementation (42.54 ms vs. 2.70 ms per image) when using 16-bit fixed-point arithmetic.
• The accuracy of the FPGA-accelerated version was 62.28% in exclusive mode, representing only a 3.26% drop from the 65.54% accuracy of the 32-bit floating-point software baseline.
• The shared-mode hardware accelerator (HW-SM) achieved 8.12 ms inference time and 62.28% accuracy, showing that exclusive mode improves performance by eliminating contention.
• BRAM utilization was the primary hardware constraint, and the framework’s software backend effectively guided CNN design to stay within resource limits.
• The precision tuning backend successfully minimized classification error by adjusting integer and fractional bit-widths across layers, maintaining high accuracy despite low-precision arithmetic.
• The framework successfully generated a working, optimized CNN accelerator from a Python-based CNN model with minimal manual intervention.