[Paper Review] Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge.

Hamed F. Langroudi, Zachariah Carmichael+2 more|arXiv (Cornell University)|Aug 6, 2019

Advanced Neural Network ApplicationsComputer Science41 references18 citations

TL;DR

Cheetah is a mixed-precision hardware-software co-design framework that enables both training and inference of deep neural networks (DNNs) using low-precision formats, including posits, fixed-point, and floating-point. It demonstrates that 16-bit posits outperform 16-bit floating-point in DNN training, and 5–8-bit posits achieve better performance and energy-delay-product trade-offs than equivalent fixed-point or floating-point formats on edge devices.

ABSTRACT

Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference only. In this paper, we propose the Cheetah framework, which supports both DNN training and inference using posits, as well as other commonly used formats. Additionally, the framework is amenable for different quantization approaches and supports mixed-precision floating point and fixed-point numerical formats. Cheetah is evaluated on three datasets: MNIST, Fashion MNIST, and CIFAR-10. Results indicate that 16-bit posits outperform 16-bit floating point in DNN training. Furthermore, performing inference with [5..8]-bit posits improves the trade-off between performance and energy-delay-product over both [5..8]-bit float and fixed-point.

Motivation & Objective

• To address the gap in low-precision DNN frameworks that support both training and inference using emerging numerical formats like posits.
• To evaluate the efficacy of different numerical formats (fixed-point, floating-point, posit) across varying bit-widths and quantization methods.
• To compare the impact of quantization approaches (rounding vs. linear) on accuracy and hardware complexity in low-precision DNNs.
• To optimize the energy-delay-product and inference accuracy trade-off for edge deployment of DNNs.
• To provide a unified co-design framework that enables systematic exploration of numerical formats and quantization strategies for edge AI.

Proposed method

• Cheetah integrates a software framework with a hardware accelerator (EMAC) to support mixed-precision computation using fixed-point, floating-point, and posit formats.
• The framework supports two quantization methods: rounding and linear quantization with shift operations, enabling flexible precision control.
• Posit formats are parameterized by word length (N) and exponent width (es), with configurations like <8,1> and <5,1> used in evaluation.
• Hardware energy-delay-product and latency are measured using FPGA-based EMACs targeting 28nm and 14nm process nodes.
• The framework evaluates DNNs on MNIST, Fashion MNIST, and CIFAR-10 using feedforward and convolutional neural networks.
• Comparative analysis is conducted across numerical formats and quantization methods, measuring accuracy degradation, energy-delay-product, and latency.

Experimental results

Research questions

• RQ1 Does the posit numerical format enable better DNN training performance compared to 16-bit floating-point?
• RQ2 Can 5–8-bit posits achieve superior performance and energy-delay-product trade-offs compared to 5–8-bit fixed-point and floating-point formats in DNN inference?
• RQ3 How do different quantization approaches (rounding vs. linear) affect accuracy and hardware complexity across numerical formats?
• RQ4 What is the optimal posit configuration (e.g., <8,1> vs. <8,0>) for minimizing energy-delay-product while maintaining accuracy?
• RQ5 Is there a consistent advantage in using mixed-precision formats across different datasets and DNN architectures?

Key findings

• 16-bit posits outperform 16-bit floating-point in DNN training, indicating posits are viable for low-precision training.
• For 5–8-bit inference, posits with es = 1 achieve 2% better average accuracy than es = 2 and 4% better than es = 0.
• Posits with es = 1 reduce energy-delay-product by 2.1× compared to es = 2 and 1.4× more than es = 0.
• Linear quantization with posits (es = 0) achieves similar accuracy to es = 1 but reduces energy-delay-product by 18%.
• The lowest accuracy degradation across datasets is achieved with posit <8,1> and <5,1> formats, outperforming equivalent fixed-point and floating-point configurations.
• Cheetah enables a 1.4× improvement in energy-delay-product and 2.1× reduction in energy-delay-product for posits with es = 1 compared to es = 2 and es = 0, respectively.