[论文解读] Long short-term memory and learning-to-learn in networks of spiking neurons

Guillaume Bellec, Darjan Salaj+3 more|arXiv (Cornell University)|Mar 26, 2018

Advanced Memory and Neural Computing被引用 164

一句话总结

通过 BPTT 和 DEEP R 训练的具有自适应神经元的 LSNN 在序列 MNIST 和 TIMIT 上达到与 LSTM 相似的性能；通过 Learning-to-Learn (L2L) 它们获得快速学习先验并实现元强化学习。

ABSTRACT

Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture). In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples. We demonstrate this for supervised learning and reinforcement learning.

研究动机与目标

• 动机：比较 RSNN 相对于 ANN 的局限性，并探索通过进化、发展和先验经验进行优化。
• 通过在 RSNN 中添加神经元自适应来引入 LSNN，以扩展短期记忆。
• 展示 BPTT 结合 DEEP R 能够训练 LSNN 以解决复杂任务。
• 展示学习到学习（L2L）使 LSNN 能迅速适应新任务。
• 展示 LSNN 的元强化学习（meta-RL）能力及其在神经形态方面的潜在意义。

提出的方法

• 引入具有两个神经元群的 LSNN 架构：常规 LIF 和自适应 LIF 神经元。
• 使用对尖峰的阻尼伪导数对 BPTT 进行训练以训练 LSNN。
• 将 BPTT 与 DEEP R 相结合进行突触重连以优化连通性。
• 在序列 MNIST 和 TIMIT 上进行评估，以便与 LSTM 和 RNN 基线比较。
• 应用学习到学习（L2L），外循环优化超参数以支持快速内部循环学习。
• 通过训练 LSNN 以执行基于奖励的导航任务来展示元强化学习。

实验结果

研究问题

• RQ1 LSNNs 是否能缩小在时序分类任务（如序列 MNIST 和 TIMIT）的性能差距，与 LSTM 网络相当？
• RQ2 将神经元自适应引入 RSNN 是否能有效扩展短期记忆以适应复杂任务？
• RQ3 学习到学习（L2L）是否能在 LSNN 中 imprint 先验，使其能够从少量样本快速学习新任务？
• RQ4 LSNN 是否能在不改变突触权重的情况下展示带奖励信号的元强化学习能力？

主要发现

• 在序列 MNIST 的1 ms 和2 ms像素呈现下，LSNN 达到 94.7% 和 96.4% 的准确度，接近 LSTM 的性能（98.5% 和 98.0%）。
• 在 TIMIT 上，LSNN 达到 33.2% 的分类错误率，低于约 40% 的平均值（来自 200 次 LSTM 试验），但高于最佳 20 次 LSTM 试验的 29.7%。
• 启用 DEEP R 的稀疏 LSNN（约 12% 连接度）可以优于全连接 LSNN，并接近 LSTM 指标。
• 学习到学习（L2L）使 LSNN 能从教师提供的少量试验中学习新的非线性函数，超过线性预测器并显示出快速的内部模型形成。
• 采用元强化学习训练的 LSNN 获得用于导航和规划的抽象知识，展示在稀疏 RSNN 中的奖励驱动学习能力。
• L2L 将平滑函数族的先验（例如非线性 TNs、正弦函数等）植入 LSNN，从而在不改变突触权重的情况下实现高效的在线学习。