[论文解读] Computing with hardware neurons: spiking or classical? Perspectives of applied Spiking Neural Networks from the hardware side

Sergei Dytckov, Masoud Daneshtalab|arXiv (Cornell University)|Feb 5, 2016

Advanced Memory and Neural Computing参考文献 26被引用 3

一句话总结

本文从硬件能效角度比较了脉冲神经网络与经典神经网络，表明尽管脉冲硬件在基础运算中的能效与经典系统相当或更优，但过长的脉冲表示导致的系统级低效仍制约性能。关键发现是，通过最小化脉冲数量的脉冲驱动应用可缩小能效差距，但当前系统仍受制于脉冲数量过多。

ABSTRACT

While classical neural networks take a position of a leading method in the machine learning community, spiking neuromorphic systems bring attention and large projects in neuroscience. Spiking neural networks were shown to be able to substitute networks of classical neurons in applied tasks. This work explores recent hardware designs focusing on perspective applications (like convolutional neural networks) for both neuron types from the energy efficiency side to analyse whether there is a possibility for spiking neuromorphic hardware to grow up for a wider use. Our comparison shows that spiking hardware is at least on the same level of energy efficiency or even higher than non-spiking on a level of basic operations. However, on a system level, spiking systems are outmatched and consume much more energy due to inefficient data representation with a long series of spikes. If spike-driven applications, minimizing an amount of spikes, are developed, spiking neural systems may reach the energy efficiency level of classical neural systems. However, in the near future, both type of neuromorphic systems may benefit from emerging memory technologies, minimizing the energy consumption of computation and memory for both neuron types. That would make infrastructure and data transfer energy dominant on the system level. We expect that spiking neurons have some benefits, which would allow achieving better energy results. Still the problem of an amount of spikes will still be the major bottleneck for spiking hardware systems.

研究动机与目标

• 评估实际应用中脉冲神经形态硬件相对于经典神经网络的能效表现。
• 识别脉冲硬件中的系统级低效，特别是由长期脉冲序列导致的数据表示问题。
• 探索通过优化脉冲使用，脉冲神经网络系统是否可实现与经典网络相当或更优的能效。
• 评估新兴存储技术在系统层面降低两类神经元能耗的潜力。
• 确定脉冲神经形态系统是否能克服当前局限，实现更广泛部署。

提出的方法

• 在基础计算操作层面，基准测试脉冲神经网络与经典神经网络的能效表现。
• 分析系统级能耗，重点关注因长期脉冲序列导致的脉冲系统数据表示低效问题。
• 评估脉冲驱动应用设计对减少总体脉冲数量及提升能效的影响。
• 评估新兴存储技术在降低神经形态系统中计算与存储能耗方面的潜力。
• 比较不同设计选择（如脉冲频率与数据编码策略）下的系统级能耗趋势。

实验结果

研究问题

• RQ1 在基础操作层面，脉冲神经形态硬件的能效与经典神经网络相比如何？
• RQ2 脉冲神经形态系统中的主要系统级能效瓶颈是什么？它们如何影响整体性能？
• RQ3 通过应用级优化减少脉冲数量，能否缩小脉冲网络与经典网络之间的能效差距？
• RQ4 新兴存储技术在多大程度上可降低脉冲与经典神经形态系统中的能耗？
• RQ5 哪些因素将决定脉冲硬件能否实现与经典硬件相当的系统级能效？

主要发现

• 在基础操作层面，脉冲硬件的能效与经典硬件相当或更优。
• 由于长期脉冲序列导致的数据表示低效，脉冲系统的系统级能耗显著高于经典系统。
• 通过最小化脉冲数量的应用设计，可使脉冲神经形态系统实现与经典系统相当的能效。
• 新兴存储技术有望降低两类神经元的计算与存储能耗，使主要能耗成本转向基础设施与数据传输。
• 尽管具备潜在优势，脉冲数量仍是限制脉冲神经形态硬件可扩展性与效率的主要瓶颈。