[论文解读] Spiking PointNet: Spiking Neural Networks for Point Clouds

Dayong Ren, Zhe Ma+5 more|arXiv (Cornell University)|Oct 10, 2023

Advanced Memory and Neural Computing被引用 16

一句话总结

引入 Spiking PointNet，是首个用于点云的 SNN 模型，采用“训练少但学习多”的框架，在单时间步训练、在多时间步推理下，达到或超过 ANN 和 vanilla SNN 在 ModelNet10/ModelNet40 上的表现，并通过膜电位扰动实现进一步提升。

ABSTRACT

Recently, Spiking Neural Networks (SNNs), enjoying extreme energy efficiency, have drawn much research attention on 2D visual recognition and shown gradually increasing application potential. However, it still remains underexplored whether SNNs can be generalized to 3D recognition. To this end, we present Spiking PointNet in the paper, the first spiking neural model for efficient deep learning on point clouds. We discover that the two huge obstacles limiting the application of SNNs in point clouds are: the intrinsic optimization obstacle of SNNs that impedes the training of a big spiking model with large time steps, and the expensive memory and computation cost of PointNet that makes training a big spiking point model unrealistic. To solve the problems simultaneously, we present a trained-less but learning-more paradigm for Spiking PointNet with theoretical justifications and in-depth experimental analysis. In specific, our Spiking PointNet is trained with only a single time step but can obtain better performance with multiple time steps inference, compared to the one trained directly with multiple time steps. We conduct various experiments on ModelNet10, ModelNet40 to demonstrate the effectiveness of Spiking PointNet. Notably, our Spiking PointNet even can outperform its ANN counterpart, which is rare in the SNN field thus providing a potential research direction for the following work. Moreover, Spiking PointNet shows impressive speedup and storage saving in the training phase.

研究动机与目标

• 推动面向 3D 点云处理的能效型 SNNs。
• 将 PointNet 适配到脉冲神经网络框架，同时解决优化与资源挑战。
• 提出一种使用单一时间步的训练方案，同时利用多步推理以获得更好性能。
• 在训练过程中通过膜电位扰动提升泛化能力。
• 在 ModelNet10/ModelNet40 上评估其有效性并与 ANN 和 vanilla SNN 进行对比。

提出的方法

• 将 PointNet 的神经元替换为 Leaky Integrate-and-Fire (LIF) 脉冲神经元。
• 使用显式迭代的 LIF 模型进行时序反向传播。
• 应用带可调 surrogate k 的替代梯度方法来管理梯度流。
• 引入 trained-less but learning-more 范式：用 1 个时间步进行训练，使用多时间步进行推理。
• 在训练中引入膜电位扰动以提升泛化能力。
• 提供优化与内存/计算收益的理论依据和实证分析。

Figure 1: The overall of the trained-less but learning-more framework. The Spiking PointNet is trained with only one single time step in the training phase, while is used with multiple time steps in the inference phase. To improve the performance of the SNN, we also add some membrane potential pertu

实验结果

研究问题

• RQ1 SNN 是否能有效应用于像 PointNet 那样的三维点云？
• RQ2 是否有可能用单一时间步来训练点云的 SNN，同时从多步推理中受益？
• RQ3 替代梯度的选择（k）如何影响大时间步的训练稳定性和性能？
• RQ4 膜电位扰动是否能提升对 1 步训练的 SNN 的泛化能力？
• RQ5 Spiking PointNet 相对于 ANN 和 vanilla SNN 基线的能耗与内存优势是什么？

主要发现

数据集	方法	训练时间步	测试时间步(1)	测试时间步(2)	测试时间步(3)	测试时间步(4)
ModelNet10	Vanilla SNN	4	89.62%	90.83%	91.05%	91.05%
ModelNet10	Ours without MPP	1	91.99%	92.43%	92.53%	92.32%
ModelNet10	Ours with MPP	1	91.66%	92.98%	92.98%	93.31%
ModelNet40	ANN	-	89.20%
ModelNet40	Vanilla SNN	4	85.59%	86.58%	86.34%	86.70%
ModelNet40	Ours without MPP	1	86.98%	87.26%	87.21%	87.13%
ModelNet40	Ours with MPP	1	87.72%	88.46%	88.25%	88.61%

• Spiking PointNet 在某些配置下可超越其对应的 ANN。
• 用单时间步训练、用多步推理可获得有竞争力的准确率，并避免大时间步训练的不稳定性。
• 使用合适的 k 的替代梯度（例如对 1 步训练的 k=5）可缓解梯度爆炸/消失问题并提升性能。
• 膜电位扰动进一步提升准确性，在 ModelNet10 的 4 次测试步中最高达到 93.31%。
• 与 ANN 的前向传播相比，该方法将能耗降低约 15 倍，同时保持较高精度；四步推理在能耗方面有显著节省。
• 该框架暗示多步推理具有集成式效应，作为针对静态点云的鲁棒性提升器。

Figure 2: Chain rule graph for gradients w.r.t. weights of SNNs