[论文解读] Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

Vladimir A. Ivanov, Konstantinos P. Michmizos|arXiv (Cornell University)|Oct 26, 2021

Advanced Memory and Neural Computing参考文献 9被引用 24

一句话总结

NALSM 使用星形胶质细胞启发的STDP调控，将液态状态机引导至近临界动态，在 MNIST 和 N-MNIST 上达到最先进的 LSM 准确率，且无需数据集特定调优，并可扩展到更大液体以实现有竞争力的性能。

ABSTRACT

The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase transition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocyte liquid state machine (NALSM) that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback to spike-timing-dependent plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific hand-tuning. With a top accuracy of 97.61% on MNIST, 97.51% on N-MNIST, and 85.84% on Fashion-MNIST, NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of brain-inspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy-efficient neuromorphic computing on the edge.

研究动机与目标

• 推动在不使用反向传播或数据集特定调优的情况下提升 LSM 的准确性。
• 利用星形胶质细胞的生物学原理来调控突触可塑性和网络动力学。
• 使 LSM 动力学接近混沌边缘，以最大化计算能力。
• 展示对更大液体大小的可扩展性以及对稀疏星形胶质细胞连接的鲁棒性。

提出的方法

• 通过嵌入一个 LIM 星形胶质细胞模型，扩展带有 STDP 的基线 LSM。
• 使用星形胶质细胞根据液体/输入活动调节 STDP 的抑制率，以实现近临界动力学。
• 从尖峰计数中定义一个代理分支因子 BF_proxy(t) 以近似临界分支因子。
• 使用液体的尖峰计数特征通过梯度下降训练输出层。
• 在 MNIST 和 N-MNIST 上评估准确性，并与 LSM 变体及基于反向传播的网络进行比较。

实验结果

研究问题

• RQ1 在不进行数据集特定调优的情况下，星形胶质细胞调制的 STDP 能否将 LSM 动力学引导至近临界的边缘-混沌行为？
• RQ2 在 MNIST 和 N-MNIST 上，NALSM 是否比基线 LSM 和 STDP 增强的 LSM 变体具有更高的准确性？
• RQ3 液体大小和神经元-星形胶质细胞连接对性能的影响是什么？
• RQ4 NALSM 的工作接近临界分支因子的程度如何，以及这与核质量和泛化之间有何关系？

主要发现

• 在 8,000 个液体下，NALSM 在 MNIST 上达到 97.61% 的最高准确率，在 N-MNIST 上达到 97.51%，在相同条件下超过基线 LSM 和 LSM+AP-STDP。
• 使用 1,000 个神经元时，NALSM 达到 96.15%（MNIST）和 96.13%（N-MNIST），优于 LSM、LSM+STDP 和 LSM+AP-STDP。
• NALSM 在神经元-星形胶质细胞连接稀疏至 10% 密度时仍保持准确性优势。
• 随着液体增大，准确性提升，在大约 8,000 个神经元时达到饱和，并且在 Fashion-MNIST（85.84%）上保持竞争力。
• 核质量（线性可分与泛化）与更高的准确性相关，支持近临界动力学对 LSM 有利。