[Paper Review] A 481pJ/decision 3.4M decision/s Multifunctional Deep In-memory Inference Processor using Standard 6T SRAM Array

Mingu Kang, Sujan K. Gonugondla+2 more|arXiv (Cornell University)|Oct 24, 2016

Advanced Memory and Neural Computing3 references22 citations

TL;DR

This paper presents a 65 nm CMOS 16KB standard 6T SRAM-based deep in-memory inference processor that achieves 481 pJ/decision at 3.4M decisions per second with up to 5.6× energy reduction and ≤1% accuracy loss across four applications (SVM, matched filter, template matching, KNN). It enables multifunctional inference via pitch-matched analog processing in the SRAM bit-cell array using sub-ranged multi-row functional reads, reconfigurable bit-line and cross-bit-line processing, and low-swing charge-sharing computation, eliminating off-chip data movement for energy efficiency.

ABSTRACT

This paper describes a multi-functional deep in-memory processor for inference applications. Deep in-memory processing is achieved by embedding pitch-matched low-SNR analog processing into a standard 6T 16KB SRAM array in 65 nm CMOS. Four applications are demonstrated. The prototype achieves up to 5.6X (9.7X estimated for multi-bank scenario) energy savings with negligible (<1%) accuracy degradation in all four applications as compared to the conventional architecture.

Motivation & Objective

• To address the high energy cost and memory bandwidth bottleneck in conventional deep learning inference architectures by embedding analog computation directly in SRAM arrays.
• To enable multifunctional inference (dot product, Manhattan distance, classification) within a single standard 6T SRAM array without altering storage density or read/write functionality.
• To achieve high energy efficiency and throughput by leveraging massively parallel in-memory processing using sub-ranged multi-row functional reads and charge-sharing computation.
• To demonstrate negligible accuracy degradation (<1%) while reducing memory access energy through reduced read cycles and in-situ analog operations.

Proposed method

• Uses pulse-width modulated word-line (PWM-WL) signals to perform multi-row functional read (MR-FR), enabling weighted sum computation across multiple rows in a single precharge cycle.
• Employs sub-ranged read by splitting 8-bit words into 4 MSB and 4 LSB columns, with charge sharing (1/16 ratio) to enable high-resolution analog computation within column pitch constraints.
• Reconfigures bit-line processing (BLP) and cross-bit-line processing (CBLP) stages to operate in dot product (DP) or Manhattan distance (MD) modes via analog multiplexers and comparators.
• Utilizes a mixed-signal capacitive multiplier with sequential bit processing, enhanced by parallel 4-bit MSB/LSB multipliers to improve throughput.
• Integrates a digital controller and pipelined architecture to enable concurrent precharge and processing, supporting 128×8-bit word access per cycle.
• Employs slow, energy-efficient single-slope ADCs with charge-sharing between two consecutive CBLP outputs to reduce power consumption.

Experimental results

Research questions

• RQ1 Can a standard 6T SRAM array be extended to perform multifunctional deep in-memory inference without modifying its storage or read/write functionality?
• RQ2 How can analog computation be embedded in SRAM bit-cell arrays while maintaining pitch-matching and linearity under process and process variation?
• RQ3 To what extent can energy efficiency be improved by reducing memory access cycles through in-situ computation in the SRAM array?
• RQ4 What is the accuracy and energy trade-off when using low-swing, charge-sharing analog computation in a 65 nm CMOS process?
• RQ5 Can the same hardware support multiple inference workloads (e.g., SVM, KNN, template matching) via reconfigurable processing stages?

Key findings

• The prototype achieves 481 pJ/decision at 3.4M decisions per second, with up to 5.6× energy reduction compared to conventional architectures and ≤1% accuracy degradation across all four applications.
• The chip reduces memory access energy by 16× due to sub-ranged multi-row functional reads, which process 128×8-bit words per access cycle instead of 8×8-bit words via column multiplexing.
• The measured maximum INL of the sub-ranged MR-FR is 0.03 LSB, ensuring high linearity in analog-to-digital conversion.
• In the DP mode, the maximum error magnitude is 5.8% of the output dynamic range; in the MD mode, it is 8.6%.
• The energy breakdown shows that the MR-FR stage contributes most to energy savings, with CORE energy reducing by 0.2 pJ per 20 mV reduction in ΔVBL.
• In a multi-bank scenario, estimated energy savings reach up to 9.7× (DP mode) and 5.4× (MD mode), with the controller energy amortized across banks.