[Paper Review] A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration

Yuhang Li, Shikuang Deng+3 more|arXiv (Cornell University)|Jun 13, 2021

Advanced Memory and Neural Computing37 references50 citations

TL;DR

The paper introduces SNN Calibration, a layer-wise calibration method that converts pretrained ANNs to low-latency, accurate SNNs with a small calibration data budget, achieving state-of-the-art results on MobileNet and RegNet on ImageNet.

ABSTRACT

Spiking Neural Network (SNN) has been recognized as one of the next generation of neural networks. Conventionally, SNN can be converted from a pre-trained ANN by only replacing the ReLU activation to spike activation while keeping the parameters intact. Perhaps surprisingly, in this work we show that a proper way to calibrate the parameters during the conversion of ANN to SNN can bring significant improvements. We introduce SNN Calibration, a cheap but extraordinarily effective method by leveraging the knowledge within a pre-trained Artificial Neural Network (ANN). Starting by analyzing the conversion error and its propagation through layers theoretically, we propose the calibration algorithm that can correct the error layer-by-layer. The calibration only takes a handful number of training data and several minutes to finish. Moreover, our calibration algorithm can produce SNN with state-of-the-art architecture on the large-scale ImageNet dataset, including MobileNet and RegNet. Extensive experiments demonstrate the effectiveness and efficiency of our algorithm. For example, our advanced pipeline can increase up to 69% top-1 accuracy when converting MobileNet on ImageNet compared to baselines. Codes are released at https://github.com/yhhhli/SNN_Calibration.

Motivation & Objective

• Motivate efficient SNN deployment by addressing conversion-induced activation mismatch between ANN and SNN.
• Propose a layer-wise calibration framework that leverages a pretrained ANN to produce accurate, low-latency SNNs.
• Analyze conversion error components and its propagation to design targeted calibration strategies.
• Offer practical pipelines (Light and Advanced) to balance accuracy, data needs, and computation.
• Demonstrate scalability to large architectures (MobileNet, RegNetX-4GF) and ImageNet-scale tasks.

Proposed method

• Formulate ANN-to-SNN conversion as a propagation problem with floor and clipping errors in spike generation.
• Introduce MMSE-based adaptive threshold to set per-layer SNN thresholds across time steps.
• Develop layer-wise calibration algorithms to adjust bias (Light Pipeline) and optionally initial membrane potential and weights (Advanced Pipeline).
• Handle BN, AvgPool, and conversion-specific components by integrating BN absorption and pool-to-convolution handling.
• Provide two practical pipelines (Light: bias-only; Advanced: bias, initial potential, and weights) with data-budgeted calibration.
• Use a deterministic loss-based calibration loop leveraging a straight-through estimator for gradient flow during weight calibration.

Experimental results

Research questions

• RQ1 How can ANN-to-SNN conversion errors be characterized and bounded across layers?
• RQ2 Can a minimal, data-efficient calibration scheme close the activation gap between ANN and SNN at very low time steps (≤256)?
• RQ3 What thresholding strategy best balances floor and clipping errors during conversion and how does MMSE perform across layers?
• RQ4 What are the trade-offs and practical gains of Light vs Advanced calibration pipelines on large-scale models and BN-containing architectures?
• RQ5 How do BN, AvgPool, and other common layers affect conversion, and can they be effectively integrated into calibration?

Key findings

• Adaptive MMSE thresholds yield lower conversion error than fixed or extreme threshold choices, especially at short simulation lengths.
• Bias calibration (Light Pipeline) consistently improves accuracy across BN configurations and thresholds, with low memory overhead.
• Potential and Weights Calibration (Advanced Pipeline) further improves accuracy, achieving notable gains on VGG-16 and MobileNet across ImageNet.
• The method enables high-accuracy conversion for MobileNet and RegNetX-4GF with T ≤ 256, outperforming several baselines.
• Advanced Pipeline with full WC can reach competitive accuracy with significantly lower latency than end-to-end training approaches.
• Energy efficiency remains high, with sparse firing and substantial energy savings on neuromorphic hardware (e.g., 69.36% energy compared to ANN in a Spiking VGG-16 case).