[论文解读] Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection

Xinhao Luo, Man Yao+3 more|arXiv (Cornell University)|Jul 30, 2024

Advanced Memory and Neural Computing被引用 5

一句话总结

提出 SpikeYOLO，一种带有 Integer Leaky Integrate-and-Fire (I-LIF) 神经元的尖峰神经网络方法，以及一个简化的基于 YOLO 的架构，在静态和神经形态数据集上实现高对象检测精度与低功耗。

ABSTRACT

Brain-inspired Spiking Neural Networks (SNNs) have bio-plausibility and low-power advantages over Artificial Neural Networks (ANNs). Applications of SNNs are currently limited to simple classification tasks because of their poor performance. In this work, we focus on bridging the performance gap between ANNs and SNNs on object detection. Our design revolves around network architecture and spiking neuron. First, the overly complex module design causes spike degradation when the YOLO series is converted to the corresponding spiking version. We design a SpikeYOLO architecture to solve this problem by simplifying the vanilla YOLO and incorporating meta SNN blocks. Second, object detection is more sensitive to quantization errors in the conversion of membrane potentials into binary spikes by spiking neurons. To address this challenge, we design a new spiking neuron that activates Integer values during training while maintaining spike-driven by extending virtual timesteps during inference. The proposed method is validated on both static and neuromorphic object detection datasets. On the static COCO dataset, we obtain 66.2% mAP@50 and 48.9% mAP@50:95, which is +15.0% and +18.7% higher than the prior state-of-the-art SNN, respectively. On the neuromorphic Gen1 dataset, we achieve 67.2% mAP@50, which is +2.5% greater than the ANN with equivalent architecture, and the energy efficiency is improved by 5.7*. Code: https://github.com/BICLab/SpikeYOLO

研究动机与目标

• 缩小用于对象检测的 ANN 与 SNN 之间的性能差距。
• 在将复杂 CNN 模块转换为 SNN 形式时减少尖峰劣化。
• 引入整数值训练以降低 SNNs 的量化误差。
• 在部署阶段启用以尖峰驱动的推理，以保持能效。

提出的方法

• 提出 SpikeYOLO，一种集成了元 SNN 模块以实现鲁棒尖峰基特征提取的 YOLOv8 的简化宏设计。
• 引入 I-LIF，一种以整数激活进行训练并在推理时转换为二进制尖峰以减少量化误差的整数值脉冲神经元。
• 在推理阶段使用扩展的虚拟时间步，将整数激活映射到以尖峰驱动的计算。
• 调整输入处理以适应静态图像和神经形态事件流，包括具有两个专门化 SNN 模块（SNN-Block-1 和 SNN-Block-2）的 SpikeYOLO 架构。
• 在静态 COCO 2017 val 和神经形态 Gen1 数据集上进行评估，并就 mAP 和能效与先前的 SNN 与 ANN 进行比较。

Figure 1 : The overall architecture of SpikeYOLO. We designed two SNN blocks, SNN-Block-1 and SNN-Block-2, and kept other architectures remain as YOLOv8. SNN-Block-1 employs standard convolution within its $\rm{ChannelConv\left(\cdot\right)}$ component, whereas SNN-Block-2 utilizes re-parameterizati

实验结果

研究问题

• RQ1 SpikeYOLO 是否能够在标准数据集和神经形态数据集上缩小 SNN 与 ANN 在对象检测上的性能差距？
• RQ2 通过 I-LIF 的整数值训练是否能将量化误差降至足以提高检测精度，同时不牺牲尖峰驱动推理？
• RQ3 为实现有效的基于 SNN 的对象检测，需要哪些体系结构改造（模块设计和检测头）？
• RQ4 时间步数（T）和最大整数激活值（D）如何影响 COCO 和 Gen1 数据集上的性能与能耗？

主要发现

• SpikeYOLO 在 COCO 2017 val 上达到 66.2% mAP@50 和 48.9% mAP@50:95，分别较先前的 SNN 提升 15.0% 和 18.7%。
• 在 Gen1 神经形态数据集上，SpikeYOLO 达到 67.2% mAP@50，超过同等架构的 ANN 2.5%，并实现约 5.7× 的能效提升。
• I-LIF 神经元通过用整数激活进行训练并通过扩展的虚拟时间步转换为尖峰，实现尖峰驱动推理，从而降低量化误差。
• 架构消融研究表明，简化 YOLO 模块并使用元 SNN 块相较于直接的 ANN-to-SNN 转换和更复杂的 SNN 设计，带来显著提升。
• 量化参数研究表明，增大最大整数值 D 常常降低量化误差，并在提高准确性方面比单纯增加时间步长 T 更有效，且对能耗有显著影响。
• SpikeYOLO 在 COCO 上取得优异结果，同时保持低于可比 ANN 模型的能耗，并在 Gen1 上展示出显著的能效提升的良好性能。

Figure 2 : Comparison of I-LIF and LIF. Binary spikes are emitted by LIF during both training and inference processes, which results in quantization errors. I-LIF emits integer values during the training process to reduce quantization errors, and converts them into binary spikes during inference to