[论文解读] Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

Nitin Rathi, Gopalakrishnan Srinivasan+2 more|arXiv (Cornell University)|May 4, 2020

Advanced Memory and Neural Computing参考文献 22被引用 164

一句话总结

该论文提出了一种混合训练方法：从人工神经网络 (ANN) 转换得到一个已转换的脉冲神经网络（SNN）并通过脉冲时序相关反向传播（STDB）进行微调，以在大幅减少时间步数和训练成本的同时实现类似的准确率。

ABSTRACT

Spiking Neural Networks (SNNs) operate with asynchronous discrete events (or spikes) which can potentially lead to higher energy-efficiency in neuromorphic hardware implementations. Many works have shown that an SNN for inference can be formed by copying the weights from a trained Artificial Neural Network (ANN) and setting the firing threshold for each layer as the maximum input received in that layer. These type of converted SNNs require a large number of time steps to achieve competitive accuracy which diminishes the energy savings. The number of time steps can be reduced by training SNNs with spike-based backpropagation from scratch, but that is computationally expensive and slow. To address these challenges, we present a computationally-efficient training technique for deep SNNs. We propose a hybrid training methodology: 1) take a converted SNN and use its weights and thresholds as an initialization step for spike-based backpropagation, and 2) perform incremental spike-timing dependent backpropagation (STDB) on this carefully initialized network to obtain an SNN that converges within few epochs and requires fewer time steps for input processing. STDB is performed with a novel surrogate gradient function defined using neuron's spike time. The proposed training methodology converges in less than 20 epochs of spike-based backpropagation for most standard image classification datasets, thereby greatly reducing the training complexity compared to training SNNs from scratch. We perform experiments on CIFAR-10, CIFAR-100, and ImageNet datasets for both VGG and ResNet architectures. We achieve top-1 accuracy of 65.19% for ImageNet dataset on SNN with 250 time steps, which is 10X faster compared to converted SNNs with similar accuracy.

研究动机与目标

• Motivate and address the high latency of ANN-SNN conversion and the high cost of training SNNs from scratch.
• Propose a hybrid training pipeline that uses converted SNNs as initialization for STDB-based fine-tuning.
• Develop a novel surrogate gradient based on spike timing to enable efficient spike-based learning.
• Demonstrate scalability to deep architectures (VGG, ResNet) on CIFAR and ImageNet with reduced time steps.

提出的方法

• Use an ANN-SNN converted network as the initialization for the SNN.
• Train the initialized SNN with spike timing dependent backpropagation (STDB) using a surrogate gradient based on neuron spike times.
• Define a surrogate gradient where the derivative of the spike activation with respect to membrane potential decays with the time since last spike (Δt).
• Employ a soft-reset LIF neuron model with discrete-time updates suitable for PyTorch implementation.
• Apply threshold balancing during conversion and keep a shared threshold per layer while learning with STDB.
• Evaluate on CIFAR-10, CIFAR-100 and ImageNet using VGG and ResNet-like architectures.

实验结果

研究问题

• RQ1 Can a converted SNN initialized from an ANN reduce the training time and latency when trained end-to-end with spike-based backpropagation?
• RQ2 Does spike timing dependent backpropagation with a spike-time surrogate gradient enable deep SNNs to achieve competitive accuracy with fewer time steps?
• RQ3 How does the hybrid approach compare to pure ANN-SNN conversion and pure spike-based training across standard datasets?
• RQ4 What are the effects of threshold handling and architectural choices (VGG, Residual nets) on performance for SNNs trained with STDB?

主要发现

• Hybrid training yields 10×–25× fewer time steps while preserving accuracy comparable to purely converted SNNs.
• Training converges in fewer than 20 epochs for common image datasets.
• On ImageNet with 250 time steps, the hybrid approach achieved 65.19% top-1 accuracy for VGG16, approaching ANN performance with much lower latency.
• Hybrid STDB training reduces spike activity by about 1.5× compared to purely converted SNNs at iso conditions, improving energy efficiency.
• Across CIFAR-10/100 and ImageNet, VGG/ResNet variants show near-ISO accuracy relative to ANNs and converted SNNs with significantly reduced time steps.
• For CIFAR-10, the VGG16-based hybrid model attains 91.13% at T=100, and 92.02% at T=200 in the reported comparisons.