[论文解读] Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture

Galen Pogoncheff, Jacob Granley+1 more|arXiv (Cornell University)|May 18, 2023

Neural dynamics and brain function被引用 5

一句话总结

该论文提出了一种生物约束的深度学习架构，将神经科学导出的组件——中心-周围拮抗、局部感受野、调谐归一化和皮层放大——整合到ResNet50主干网络中，以提升与初级视觉皮层（V1）神经活动的一致性。通过系统性地整合这些机制，该模型在Brain-Score基准上实现了最先进性能，在V1整体得分、预测性得分和属性得分方面均创下新SOTA。

ABSTRACT

Convolutional neural networks (CNNs) have recently emerged as promising models of the ventral visual stream, despite their lack of biological specificity. While current state-of-the-art models of the primary visual cortex (V1) have surfaced from training with adversarial examples and extensively augmented data, these models are still unable to explain key neural properties observed in V1 that arise from biological circuitry. To address this gap, we systematically incorporated neuroscience-derived architectural components into CNNs to identify a set of mechanisms and architectures that comprehensively explain neural activity in V1. We show drastic improvements in model-V1 alignment driven by the integration of architectural components that simulate center-surround antagonism, local receptive fields, tuned normalization, and cortical magnification. Upon enhancing task-driven CNNs with a collection of these specialized components, we uncover models with latent representations that yield state-of-the-art explanation of V1 neural activity and tuning properties. Our results highlight an important advancement in the field of NeuroAI, as we systematically establish a set of architectural components that contribute to unprecedented explanation of V1. The neuroscience insights that could be gleaned from increasingly accurate in-silico models of the brain have the potential to greatly advance the fields of both neuroscience and artificial intelligence.

研究动机与目标

• 为解决当前CNN在对象识别任务中表现优异但无法解释V1关键神经特性的问题。
• 识别并整合生物上合理的架构组件，以增强深度神经网络与大脑之间的相似性。
• 系统评估每项神经科学启发组件对V1表征准确性的贡献。
• 开发一种神经约束的深度学习模型，使其比以往最先进模型更全面地解释V1调谐特性与神经活动。
• 证明基于神经科学启发的架构设计可显著提升与大脑对齐的表征学习效果，超越仅依赖数据增强或对抗训练的改进。

提出的方法

• 设计新型神经网络层，通过空间调谐的卷积滤波器模拟中心-周围拮抗。
• 通过空间受限卷积实现局部感受野，以模拟皮层柱状组织结构。
• 实现调谐除法归一化层，根据局部活动调节神经反应增益，模拟侧向抑制。
• 通过输入图像的极坐标变换实现皮层放大，将中央视野映射到更高分辨率区域。
• 在标准ResNet50架构中引入这些组件，构建神经约束的深度网络。
• 使用ImageNet和对抗性损坏的ImageNet数据联合训练最终模型，以评估其鲁棒性与泛化能力。

Figure 1: Design patterns of neuro-constrained architectural components. A) Difference of Gaussian implements a center-surround receptive field. B) Local receptive fields of two neurons without weight sharing. C) Tuned divisive normalization inhibits each feature map by a Gaussian-weighted average o

实验结果

研究问题

• RQ1 哪些生物启发的架构组件最显著地提升了深度神经网络表征与灵长类V1神经活动的一致性？
• RQ2 中心-周围拮抗与调谐归一化在建模人工V1神经元的侧向调制与调谐特性方面分别发挥何种作用？
• RQ3 与其它架构组件相比，皮层放大在提升V1表征准确性方面有多大贡献？
• RQ4 对抗性训练能否放大特定架构组件（如中心-周围卷积）在解释神经反应方差方面的优势？
• RQ5 多种神经约束组件的组合是否在实现更优的脑-模型对齐方面具有协同效应，优于单一组件？

主要发现

• 神经约束的ResNet50模型在V1整体得分上达到新的SOTA，显著超越以往模型。
• 该模型在V1整体得分为0.621，预测性得分为0.432，属性得分为0.842，分别显著超过先前SOTA的0.594、0.409和0.816。
• 皮层放大是影响最显著的组件，即使略微降低了ImageNet准确率，仍大幅提升了V1对齐效果。
• 中心-周围拮抗的收益具有训练数据依赖性：仅在使用对抗性损坏数据训练时，才显著改善侧向调制调谐特性。
• 调谐归一化是唯一能提升模型鲁棒性并促进调谐特性对齐的组件。
• 消融实验确认，多个组件的组合产生了协同增益，完整模型优于任何单一组件的变体。

Figure 2: ResNet50 layer 1, supplemented with neuro-constrained architectural components. Throughout the the modified layer 1, primary visual cortex ( V1 )activity is modeled with cortical magnification, center-surround convolutions, tuned normalization, and local receptive field layers. Layer 1 out