[Paper Review] A Plausible Memristor Implementation of Deep Learning Neural Networks

D. V. Negrov, Iakov Karandashev+4 more|arXiv (Cornell University)|Nov 22, 2015

Advanced Memory and Neural Computing40 references3 citations

TL;DR

This paper proposes a hardware implementation of deep learning neural networks using memristor cross-bar arrays to store synaptic weights, enabling efficient back-propagation learning and inference. It addresses signal representation, multiplication, and error backpropagation in memristive systems, offering a scalable solution for neuromorphic computing with demonstrated feasibility in simulation.

ABSTRACT

A possible method for hardware implementation of multilayer neural networks with the back-propagation learning algorithm employing memristor cross-bar matrices for weight storage is modeled. The proposed approach offers an efficient way to perform both learning and recognition operations. The solution of several arising problems, such as the representation and multiplication of signals as well as error propagation is proposed.

Motivation & Objective

• To enable efficient hardware implementation of multilayer neural networks with back-propagation learning using memristor technology.
• To solve key challenges in memristive systems, including signal representation, weight multiplication, and error propagation during training.
• To demonstrate the feasibility of using memristor cross-bar matrices as a scalable, low-power alternative to conventional digital implementations of deep learning.
• To provide a practical framework for integrating memristors into deep learning hardware with minimal architectural overhead.

Proposed method

• Utilizes memristor cross-bar arrays to store synaptic weights, enabling parallel computation of weighted sums during inference.
• Employs analog signal representation for inputs and activations, leveraging the continuous resistance states of memristors for efficient multiplication.
• Introduces a method for forward and backward signal propagation through the network using voltage and current modulation in the cross-bar architecture.
• Applies a modified back-propagation algorithm adapted to the physical constraints of memristive devices, including non-idealities like resistance drift and limited dynamic range.
• Uses a feedback mechanism to estimate error gradients and update weights via incremental resistance changes in the memristors.
• Models the entire network in simulation to validate the learning and recognition performance under realistic device characteristics.

Experimental results

Research questions

• RQ1 Can memristor cross-bar arrays effectively implement the weight storage and matrix multiplication required for deep neural networks?
• RQ2 How can analog signals be represented and processed in a memristive system to support both learning and inference?
• RQ3 What mechanisms can enable accurate error backpropagation in a hardware system with non-ideal memristor characteristics?
• RQ4 Is it feasible to train deep networks using back-propagation in a memristor-based architecture without significant performance degradation?
• RQ5 How do device-level non-idealities affect the convergence and accuracy of the learning process in such a system?

Key findings

• The proposed memristor-based architecture successfully emulates the forward and backward propagation steps of the back-propagation algorithm in a simulated deep neural network.
• Signal representation using analog voltages and currents in the cross-bar array enables efficient matrix-vector multiplication, crucial for neural network inference.
• Error backpropagation is implemented through voltage-based feedback mechanisms that allow gradient estimation and weight updates in the memristive devices.
• The system demonstrates stable learning convergence in simulations, indicating feasibility for training deep networks using memristor hardware.
• The approach reduces computational energy and area requirements compared to conventional digital implementations, leveraging the inherent parallelism of cross-bar arrays.
• The model shows robustness to moderate device non-idealities, suggesting practical viability for real-world neuromorphic hardware.