[论文解读] Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchy

Siavash Golkar, Tiberiu Teşileanu+3 more|arXiv (Cornell University)|Oct 27, 2022

Neural dynamics and brain function被引用 5

一句话总结

该论文提出BioCCPC，一种生物上合理的预测编码变体，通过基于潜在空间去相关性的约束优化框架，替代对称权重和独立的值/误差神经元。在协方差约束下推导预测编码目标的上界后，作者表明该算法可自然映射至具有非赫布可塑性的多 compartment 锥体神经元，无需一一对应连接或信号复用，同时保持性能。

ABSTRACT

Predictive coding has emerged as an influential normative model of neural computation, with numerous extensions and applications. As such, much effort has been put into mapping PC faithfully onto the cortex, but there are issues that remain unresolved or controversial. In particular, current implementations often involve separate value and error neurons and require symmetric forward and backward weights across different brain regions. These features have not been experimentally confirmed. In this work, we show that the PC framework in the linear regime can be modified to map faithfully onto the cortical hierarchy in a manner compatible with empirical observations. By employing a disentangling-inspired constraint on hidden-layer neural activities, we derive an upper bound for the PC objective. Optimization of this upper bound leads to an algorithm that shows the same performance as the original objective and maps onto a biologically plausible network. The units of this network can be interpreted as multi-compartmental neurons with non-Hebbian learning rules, with a remarkable resemblance to recent experimental findings. There exist prior models which also capture these features, but they are phenomenological, while our work is a normative derivation. The network we derive does not involve one-to-one connectivity or signal multiplexing, which the phenomenological models required, indicating that these features are not necessary for learning in the cortex. The normative nature of our algorithm in the simplified linear case allows us to prove interesting properties of the framework and analytically understand the computational role of our network's components. The parameters of our network have natural interpretations as physiological quantities in a multi-compartmental model of pyramidal neurons, providing a concrete link between PC and experimental measurements carried out in the cortex.

研究动机与目标

• 解决预测编码（PC）与皮层神经生理学之间的不一致，特别是关于对称权重、独立的误差/值神经元以及一一对应连接的问题。
• 开发一种规范性（而非现象学）模型，以解释近期关于多 compartment 锥体神经元计算的实验发现。
• 从第一性原理出发推导出一种生物上合理的学习算法，其性能与标准PC相当，同时与已知的皮层回路一致。
• 表明信号复用和对称反馈权重并非分层皮层回路中有效学习的必要条件。

提出的方法

• 在预测编码框架中对隐藏层表示的协方差引入受去相关性启发的不等式约束。
• 在该约束下推导预测编码目标的解析上界，将优化问题转化为可处理的形式。
• 使用随机梯度下降优化上界，得到一种使用独立前向与反向权重的学习算法。
• 将所得算法映射至多 compartment 神经元模型，其中顶端和基底 compartment 的电导对应于网络参数。
• 表明所推导的算法无需对称权重或值单元与误差单元之间的一一对应连接。
• 将网络参数解释为生理量（如膜电导和漏电导率），从而将算法与实验测量直接关联。

实验结果

研究问题

• RQ1 预测编码能否被重新表述，以消除皮层回路中对称前向与反向权重的需求？
• RQ2 是否存在一种规范性推导的预测编码，能自然产生多 compartment 锥体神经元中的非赫布可塑性？
• RQ3 是否可以放宽值神经元与误差神经元之间一一对应连接的要求而不损失性能？
• RQ4 信号复用是否为分层皮层网络中有效学习的必要条件？
• RQ5 基于约束的预测编码重构能否产生一种生物上合理的学习规则，与钙平台电位及顶端-基底 compartment 相互作用的实证发现一致？

主要发现

• 所提出的BioCCPC算法在使用非对称权重且无一一对应连接的情况下，在线性区域内实现了与标准预测编码相当的性能。
• 该算法可自然映射至多 compartment 锥体神经元模型，其中顶端与基底 compartment 的电导对应于网络参数。
• 所推导的学习规则为非赫布型，与实验观察一致：钙平台电位驱动基底突触可塑性。
• 协方差约束起到了强正则化作用，如训练与验证损失曲线所示，过拟合程度极低。
• 该模型无需信号复用或对称反馈权重，表明这些特征并非皮层学习的本质要素。
• 网络参数具有直接的生理学解释，从而在预测编码与实验神经生理学之间建立了明确的联系。