[论文解读] One step back, two steps forward: interference and learning in recurrent neural networks

Chen Beer, Omri Barak|arXiv (Cornell University)|May 24, 2018

Neural dynamics and brain function参考文献 34被引用 4

一句话总结

本文研究了在生物上合理的学习规则（如LMS）下，循环神经网络中序列学习如何因试次间的干扰而受损。结果表明，LMS由于试次相关性过高，导致严重遗忘，而FORCE学习则通过利用二阶统计量避免了这一问题，从而造成LMS收敛速度慢几个数量级，揭示了在稳定动力学记忆形成的学习规则设计中存在关键权衡。

ABSTRACT

Artificial neural networks, trained to perform cognitive tasks, have recently been used as models for neural recordings from animals performing these tasks. While some progress has been made in performing such comparisons, the evolution of network dynamics throughout learning remains unexplored. This is paralleled by an experimental focus on recording from trained animals, with few studies following neural activity throughout training. In this work, we address this gap in the realm of artificial networks by analyzing networks that are trained to perform memory and pattern generation tasks. The functional aspect of these tasks corresponds to dynamical objects in the fully trained network - a line attractor or a set of limit cycles for the two respective tasks. We use these dynamical objects as anchors to study the effect of learning on their emergence. We find that the sequential nature of learning has major consequences for the learning trajectory and its final outcome. Specifically, we show that Least Mean Squares (LMS), a simple gradient descent suggested as a biologically plausible version of the FORCE algorithm, is constantly obstructed by forgetting, which is manifested as the destruction of dynamical objects from previous trials. The degree of interference is determined by the correlation between different trials. We show which specific ingredients of FORCE avoid this phenomenon. Overall, this difference results in convergence that is orders of magnitude slower for LMS. Learning implies accumulating information across multiple trials to form the overall concept of the task. Our results show that interference between trials can greatly affect learning, in a learning rule dependent manner. These insights can help design experimental protocols that minimize such interference, and possibly infer underlying learning rules by observing behavior and neural activity throughout learning.

研究动机与目标

• 理解序列学习如何影响训练后的循环网络中固定点和极限环等动力学结构的出现。
• 研究为何某些学习规则（如LMS）尽管在生物学上合理，却无法在试次间稳定记忆表征。
• 识别FORCE算法中防止干扰并实现稳定学习的具体组件。
• 探索试次顺序与学习率如何影响记忆与模式生成任务中的收敛性与稳定性。

提出的方法

• 研究采用基于发放率的循环神经网络，其连接矩阵随机，输出权重通过FORCE和LMS算法进行训练。
• 通过逆相关矩阵和线性稳定性分析，识别并追踪训练过程中的动力学对象——记忆任务中的固定点和模式生成任务中的极限环。
• 网络在两项任务上进行训练：简单记忆任务（在延迟后输出刺激值）和带模式生成的记忆任务（在输入后生成正弦波）。
• 通过操纵二阶统计量（P矩阵），隔离历史依赖信息在防止干扰中的作用。
• 测试学习率与试次顺序的影响，以评估控制能力和收敛动力学。
• 采用基于奖励的赫布学习作为对照，以验证关于干扰与收敛性的发现。

实验结果

研究问题

• RQ1 为何LMS学习在序列学习任务中无法在试次间维持稳定的动力学对象？
• RQ2 FORCE算法中哪些特定组件可防止试次间的干扰，其与LMS有何不同？
• RQ3 连续试次之间的相关性如何影响遗忘程度与学习效率？
• RQ4 学习率在多大程度上影响学习过程的稳定性和收敛性？
• RQ5 重新排序试次是否能减少LMS训练网络中的干扰并提升学习性能？

主要发现

• LMS学习在试次间遭受严重干扰，导致先前形成的固定点和极限环因刺激间高度相关而消失。
• FORCE算法通过保持二阶统计量（P矩阵）避免了此类干扰，而LMS无法维持这一特性。
• 在FORCE训练中删除试次间的二阶信息会重现LMS行为，证实P矩阵的保持对稳定学习至关重要。
• 由于动力学对象反复被破坏并重建，LMS的收敛速度比FORCE慢几个数量级。
• 较慢的学习率会加剧控制问题，导致网络偏离目标流形，增加训练过程中的误差。
• 基于奖励的赫布学习表现出与LMS相似的行为，仅收敛至最后一个刺激，表明干扰敏感规则存在共同的失效模式。