[Paper Review] Dynamic Stripes: Exploiting the Dynamic Precision Requirements of Activation Values in Neural Networks

Alberto Delmás, Patrick Judd+2 more|arXiv (Cornell University)|Jun 1, 2017

Advanced Neural Network Applications2 references15 citations

TL;DR

Dynamic Stripes introduces a runtime precision adaptation technique for deep neural network accelerators, dynamically reducing bit-serial computation by detecting the minimal required precision per group of 16 activations. By exploiting variable precision at a finer granularity than per-layer profiling, it achieves a 41% performance gain over the baseline Stripes design, with speedups up to 3.28x over DaDianNao on modern CNNs.

ABSTRACT

Stripes is a Deep Neural Network (DNN) accelerator that uses bit-serial computation to offer performance that is proportional to the fixed-point precision of the activation values. The fixed-point precisions are determined a priori using profiling and are selected at a per layer granularity. This paper presents Dynamic Stripes, an extension to Stripes that detects precision variance at runtime and at a finer granularity. This extra level of precision reduction increases performance by 41% over Stripes.

Motivation & Objective

• To address the inefficiency of fixed per-layer precision in DNN accelerators like Stripes, which underutilizes dynamic precision variation.
• To explore whether finer-grained, runtime-optimized precision selection—down to groups of 16 activations—can significantly improve performance.
• To design and evaluate a hardware-software co-design that detects minimal required precision (nH and nL) per activation group in real time.
• To compare the performance of dynamic precision adaptation against static profiling and alternative optimization strategies like Pragmatic’s 1-bit detection.

Proposed method

• The system uses a dispatcher to detect the most significant bit (nH) and least significant bit (nL) position with 1s in each group of 16 activations using OR-based leading-1 and trailing-1 detection circuits.
• A 4-bit offset encoder transmits nH and nL values to processing tiles, enabling bit-serial computation starting from nH and ending at nL.
• Each processing tile uses a modified Serial Inner-Product Unit (SIP) with a right-bit shifter controlled by the nH/nL offset to align partial products correctly.
• The system employs a counter and comparator to signal the end of processing per group via an End-of-Group (EOG) wire, ensuring synchronization across tiles.
• The design supports per-group precision adjustment without requiring full-width data paths, minimizing area and bandwidth overhead.
• The approach is evaluated using a cycle-accurate simulator extended to support dynamic precision detection, with results compared against Stripes and DaDianNao.

Experimental results

Research questions

• RQ1 Can dynamic, per-group precision adaptation at the 16-activation level significantly improve performance over per-layer precision profiling in DNN accelerators?
• RQ2 How much performance gain can be achieved by detecting and exploiting the actual dynamic precision requirements of activation values at runtime?
• RQ3 How does dynamic precision reduction compare to alternative optimization strategies like Pragmatic’s 1-bit detection or fixed-precision computation?
• RQ4 What is the hardware cost and area overhead of supporting dynamic precision detection at the group level?
• RQ5 Can the proposed method be effectively combined with value-based optimizations, such as truncating less significant 1-bits, to further enhance performance?

Key findings

• Dynamic Stripes achieves a 41% performance improvement over the baseline Stripes accelerator by enabling per-group precision adaptation at the 16-activation level.
• The speedup over DaDianNao ranges from 1.27x to 3.28x across different networks, with a geometric mean of 2.61x.
• On AlexNet, the combination of dynamic precision and value-based optimization (e.g., truncating less significant 1-bits) achieves a 4.83x speedup with full-range shifters.
• The method reduces computation cycles by eliminating redundant bit-serial operations on unused bits, especially in activation values with sparse 1-bits.
• The hardware extension adds only 5 wires per 16-activation group (4 for offset, 1 for EOG), resulting in minimal area and bandwidth overhead.
• The performance gain is most pronounced in networks like VGG-S and GoogLeNet, where dynamic precision adaptation enables significant reduction in computation cycles.