[Paper Review] Spike time displacement based error backpropagation in convolutional spiking neural networks

Maryam Mirsadeghi, Majid Shalchian+2 more|arXiv (Cornell University)|Aug 31, 2021

Advanced Memory and Neural Computing4 citations

TL;DR

This paper proposes an extension of the STiDi-BP algorithm for training deep convolutional spiking neural networks (CSNNs) with single-spike temporal coding, avoiding recursive backpropagation by using spike time displacement for error backpropagation. It achieves 99.2% accuracy on MNIST and 92.8% on Fashion-MNIST, while introducing a binary-weights variant that reduces memory and computation with only a 0.6–0.8% accuracy drop.

ABSTRACT

We recently proposed the STiDi-BP algorithm, which avoids backward recursive gradient computation, for training multi-layer spiking neural networks (SNNs) with single-spike-based temporal coding. The algorithm employs a linear approximation to compute the derivative of the spike latency with respect to the membrane potential and it uses spiking neurons with piecewise linear postsynaptic potential to reduce the computational cost and the complexity of neural processing. In this paper, we extend the STiDi-BP algorithm to employ it in deeper and convolutional architectures. The evaluation results on the image classification task based on two popular benchmarks, MNIST and Fashion-MNIST datasets with the accuracies of respectively 99.2% and 92.8%, confirm that this algorithm has been applicable in deep SNNs. Another issue we consider is the reduction of memory storage and computational cost. To do so, we consider a convolutional SNN (CSNN) with two sets of weights: real-valued weights that are updated in the backward pass and their signs, binary weights, that are employed in the feedforward process. We evaluate the binary CSNN on two datasets of MNIST and Fashion-MNIST and obtain acceptable performance with a negligible accuracy drop with respect to real-valued weights (about $0.6%$ and $0.8%$ drops, respectively).

Motivation & Objective

• To extend the STiDi-BP algorithm, originally for fully-connected SNNs, to deeper and convolutional architectures with single-spike-based temporal coding.
• To eliminate recursive backward gradient computation in backpropagation by using spike time displacement for local error computation.
• To reduce memory and computational costs in SNNs through a binary-weight CSNN variant with sign-based weights in inference and real-valued weights in learning.
• To enable direct, end-to-end training of deep CSNNs with temporal coding, avoiding conversion from pre-trained ANNs.
• To demonstrate that binary-weight CSNNs can achieve near-competitive performance with minimal accuracy degradation for energy-efficient neuromorphic hardware.

Proposed method

• Adapts the STiDi-BP algorithm to convolutional layers by computing desired firing times via presynaptic spike time displacement, enabling local error computation without recursive backpropagation.
• Uses a linear approximation of the derivative of spike latency with respect to membrane potential to compute gradients efficiently.
• Employs piecewise linear postsynaptic potentials and ReLU-based PSP models to simplify neural dynamics and reduce computational complexity.
• Introduces a binary-weights CSNN (BCSNN) where forward pass uses binary weights (sign of real-valued weights), while backward pass updates real-valued weights via gradient descent.
• Applies a single full-precision scaling factor per layer or filter to maintain accuracy while reducing memory footprint by 32× compared to 32-bit floating-point weights.
• Uses max-pooling based on first-spike timing within receptive fields, which is natively compatible with temporal coding and reduces computational load.

Figure 1 : The piecewise linear postsynaptic potential caused by the presynaptic spike time $t_{j}$ .

Experimental results

Research questions

• RQ1 Can the STiDi-BP algorithm be effectively extended to train deep and convolutional spiking neural networks with single-spike temporal coding?
• RQ2 Does avoiding recursive backpropagation via spike time displacement reduce computational and memory costs in deep SNNs compared to standard BP?
• RQ3 Can a binary-weight CSNN trained with STiDi-BP achieve high accuracy with minimal performance drop while enabling efficient hardware deployment?
• RQ4 How does the performance of the proposed CSNN compare to existing state-of-the-art SNNs on standard benchmarks like MNIST and Fashion-MNIST?
• RQ5 To what extent does the use of temporal single-spike coding and first-spike pooling simplify neural processing and reduce computational overhead?

Key findings

• The proposed STiDi-BP-based CSNN achieves 99.2% accuracy on the MNIST dataset, demonstrating effective training of deep convolutional SNNs with single-spike coding.
• On the more complex Fashion-MNIST dataset, the CSNN reaches 92.8% accuracy, confirming its capability on challenging image classification tasks.
• The binary-weights variant (BCSNN) achieves only a 0.6% accuracy drop on MNIST and 0.8% on Fashion-MNIST compared to the real-valued CSNN, showing negligible performance loss.
• The BCSNN reduces memory usage by 32× and replaces floating-point multipliers with simple increment/decrement operations, enabling efficient neuromorphic hardware deployment.
• The method avoids recursive gradient computation, reducing space complexity to O(N) per layer compared to O(NT) in rate-coding schemes, significantly lowering memory and computational demands.
• First-spike-based max-pooling is naturally supported and computationally efficient, aligning with the event-driven nature of SNNs and further reducing processing cost.

Figure 3 : The mean required number of spikes in the input, convolutional, hidden, and total layers in R-CSNN.