[Paper Review] SLAYER: Spike Layer Error Reassignment in Time

Sumit Bam Shrestha, Garrick Orchard|arXiv (Cornell University)|Sep 5, 2018

Seismic Imaging and Inversion Techniques473 citations

TL;DR

SLAYER introduces a general backpropagation mechanism for learning synaptic weights and axonal delays in spiking neural networks, using temporal credit assignment to backpropagate errors, and provides a GPU-accelerated implementation that achieves state-of-the-art performance on several datasets compared to existing SNN methods and ANN-to-SNN conversions.

ABSTRACT

Configuring deep Spiking Neural Networks (SNNs) is an exciting research avenue for low power spike event based computation. However, the spike generation function is non-differentiable and therefore not directly compatible with the standard error backpropagation algorithm. In this paper, we introduce a new general backpropagation mechanism for learning synaptic weights and axonal delays which overcomes the problem of non-differentiability of the spike function and uses a temporal credit assignment policy for backpropagating error to preceding layers. We describe and release a GPU accelerated software implementation of our method which allows training both fully connected and convolutional neural network (CNN) architectures. Using our software, we compare our method against existing SNN based learning approaches and standard ANN to SNN conversion techniques and show that our method achieves state of the art performance for an SNN on the MNIST, NMNIST, DVS Gesture, and TIDIGITS datasets.

Motivation & Objective

• Address the non-differentiability of spike generation to enable gradient-based training of SNNs.
• Learn both synaptic weights and axonal delays within a unified backpropagation framework.
• Implement a temporal credit assignment policy to propagate error through time across layers.
• Provide a GPU-accelerated software implementation for training fully connected and CNN architectures.
• Compare SLAYER against existing SNN learning methods and ANN-to-SNN conversion approaches on multiple datasets.

Proposed method

• Propose a backpropagation mechanism that accommodates non-differentiable spike functions in SNNs.
• Jointly learn synaptic weights and axonal delays to optimize spike-based representations.
• Employ a temporal credit assignment policy to propagate error signals to preceding layers in time.
• Develop GPU-accelerated software capable of training both fully connected and convolutional networks.
• Conduct empirical comparisons with existing SNN learning methods and ANN-to-SNN conversion techniques.

Experimental results

Research questions

• RQ1 Can backpropagation be effectively applied to SNNs by learning both weights and delays despite non-differentiable spike functions?
• RQ2 How does SLAYER compare to existing SNN learning approaches and ANN-to-SNN conversion methods in accuracy and learning efficiency?
• RQ3 What is the impact of learning axonal delays on performance across standard SNN benchmarks?
• RQ4 Does the GPU-accelerated implementation scale to convolutional architectures and datasets like MNIST, NMNIST, DVS Gesture, and TIDIGITS?

Key findings

• SLAYER achieves state-of-the-art performance for SNNs on MNIST, NMNIST, DVS Gesture, and TIDIGITS within the studied setup.
• The method enables effective learning by integrating weight and delay updates with a temporal credit assignment strategy.
• The authors provide a GPU-accelerated implementation and demonstrate its applicability to both fully connected and CNN architectures.
• Comparisons show favorable performance against existing SNN learning methods and ANN-to-SNN conversion techniques.