[Paper Review] A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference

Manuel Le Gallo, Riduan Khaddam-Aljameh+27 more|arXiv (Cornell University)|Dec 6, 2022

Advanced Memory and Neural Computing18 citations

TL;DR

This paper presents a 64-core mixed-signal in-memory computing chip based on phase-change memory (PCM) that performs deep neural network inference entirely on-chip, achieving near-software-equivalent accuracy. By integrating 64 PCM-based AIMC cores with on-chip digital processing for activation functions and inter-core communication, the chip delivers 63.1 TOPS throughput and 9.76 TOPS/W energy efficiency for 8-bit MVMs.

ABSTRACT

The need to repeatedly shuttle around synaptic weight values from memory to processing units has been a key source of energy inefficiency associated with hardware implementation of artificial neural networks. Analog in-memory computing (AIMC) with spatially instantiated synaptic weights holds high promise to overcome this challenge, by performing matrix-vector multiplications (MVMs) directly within the network weights stored on a chip to execute an inference workload. However, to achieve end-to-end improvements in latency and energy consumption, AIMC must be combined with on-chip digital operations and communication to move towards configurations in which a full inference workload is realized entirely on-chip. Moreover, it is highly desirable to achieve high MVM and inference accuracy without application-wise re-tuning of the chip. Here, we present a multi-core AIMC chip designed and fabricated in 14-nm complementary metal-oxide-semiconductor (CMOS) technology with backend-integrated phase-change memory (PCM). The fully-integrated chip features 64 256x256 AIMC cores interconnected via an on-chip communication network. It also implements the digital activation functions and processing involved in ResNet convolutional neural networks and long short-term memory (LSTM) networks. We demonstrate near software-equivalent inference accuracy with ResNet and LSTM networks while implementing all the computations associated with the weight layers and the activation functions on-chip. The chip can achieve a maximal throughput of 63.1 TOPS at an energy efficiency of 9.76 TOPS/W for 8-bit input/output matrix-vector multiplications.

Motivation & Objective

• To overcome energy inefficiency in DNN inference caused by repeated weight movement between memory and processing units.
• To enable end-to-end on-chip inference by integrating analog in-memory computing (AIMC) with on-chip digital operations and communication.
• To achieve high inference accuracy and energy efficiency without application-specific retuning, using non-volatile PCM for dense, persistent weight storage.
• To scale AIMC beyond single-core or small-network implementations by enabling multi-core, multi-layer DNN workloads on a single chip.

Proposed method

• Design and fabrication of a 14-nm CMOS chip with backend-integrated phase-change memory (PCM) for non-volatile synaptic weight storage.
• Implementation of 64 independent 256x256 AIMC cores, each performing matrix-vector multiplication (MVM) directly in memory using PCM conductance states.
• Use of a custom on-chip communication network to interconnect cores and enable distributed MVM execution across multiple cores for large layers.
• Integration of on-chip digital units (LDPU and GDPU) to perform activation functions (e.g., ReLU, tanh) and data conversions (PWM, ADC, DAC).
• Application of a conductance mapping scheme using Eq. (1) to map network weights to PCM cell conductance, with core-specific $G_{\text{max}}$ constrained by ADC saturation and current limits.
• Development of a functional hardware-aware simulation model incorporating input quantization (8-bit PWM), ADC quantization (12-bit), LDPU/GDPU data conversion (8-bit), and measured per-core weight noise via polynomial fitting.

Experimental results

Research questions

• RQ1 Can a multi-core, fully integrated AIMC chip with non-volatile PCM achieve end-to-end DNN inference with near-software-equivalent accuracy?
• RQ2 What is the maximum achievable throughput and energy efficiency of a PCM-based AIMC chip when all MVM and digital operations are performed on-chip?
• RQ3 How does on-chip weight noise and non-idealities in PCM and ADC components affect inference accuracy across multiple cores?
• RQ4 Can large DNNs (e.g., ResNet-9, LSTM) be efficiently mapped across 64 cores without off-chip weight buffers or layer-by-layer reprogramming?
• RQ5 What is the impact of core-specific $G_{\text{max}}$ scaling on MVM accuracy and energy efficiency in a multi-core PCM architecture?

Key findings

• The chip achieves a peak throughput of 63.1 TOPS for 8-bit input/output matrix-vector multiplications, with an energy efficiency of 9.76 TOPS/W.
• Near-software-equivalent inference accuracy is demonstrated for ResNet-9 and LSTM-based image captioning networks, with no application-specific retuning.
• The chip supports full on-chip execution of both MVMs and digital activation functions (e.g., ReLU, tanh) via integrated LDPU and GDPU units.
• Weight noise is modeled as a core-specific, weight-dependent Gaussian distribution, with standard deviation fitted via polynomial functions of measured per-core errors.
• For ResNet-9 using TDP PCM, $G_{\text{max}}$ values up to 160 are used, enabling higher conductance range utilization and improved MVM accuracy.
• Power measurements show that dynamic power is dominated by ADC and digital supplies, with total energy computed as the sum of static and dynamic contributions across all monitored supplies.