[论文解读] The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes

Alessandro T. Gifford, B. Lahner+7 more|arXiv (Cornell University)|Jan 9, 2023

Cell Image Analysis Techniques被引用 22

一句话总结

2023 年 Algonauts 挑战赛通过使用 Natural Scenes Dataset（NSD）以及公开榜单来比较 ~73,000 张自然场景的 fMRI 反应，从而推动建立视觉大脑的计算模型。

ABSTRACT

The sciences of biological and artificial intelligence are ever more intertwined. Neural computational principles inspire new intelligent machines, which are in turn used to advance theoretical understanding of the brain. To promote further exchange of ideas and collaboration between biological and artificial intelligence researchers, we introduce the 2023 installment of the Algonauts Project challenge: How the Human Brain Makes Sense of Natural Scenes (http://algonauts.csail.mit.edu). This installment prompts the fields of artificial and biological intelligence to come together towards building computational models of the visual brain using the largest and richest dataset of fMRI responses to visual scenes, the Natural Scenes Dataset (NSD). NSD provides high-quality fMRI responses to ~73,000 different naturalistic colored scenes, making it the ideal candidate for data-driven model building approaches promoted by the 2023 challenge. The challenge is open to all and makes results directly comparable and transparent through a public leaderboard automatically updated after each submission, thus allowing for rapid model development. We believe that the 2023 installment will spark symbiotic collaborations between biological and artificial intelligence scientists, leading to a deeper understanding of the brain through cutting-edge computational models and to novel ways of engineering artificial intelligent agents through inductive biases from biological systems.

研究动机与目标

• 促进生物智能研究者与人工智能研究者之间的合作。
• 促进使用大规模 fMRI 数据进行视觉大脑的数据驱动建模。
• 提供一个透明的、公开的基准，用于比较自然场景处理的计算模型。

提出的方法

• 利用 Natural Scenes Dataset (NSD)，其中包含对约 ~73,000 张自然场景的高质量 fMRI 反应。
• 构建与 NSD 数据一致的视觉大脑计算模型。
• 使用一个在每次提交后自动更新的公开榜单，以实现透明、快速的模型开发。
• 通过促进将生物系统的归纳偏置引入到 AI 模型，鼓励共生式合作。

实验结果

研究问题

• RQ1 如何训练计算模型以利用 NSD 预测人类对自然场景的 fMRI 反应？
• RQ2 从生物视觉获得的哪些归纳偏置可以提升人工模型在自然场景图像理解上的表现？
• RQ3 公开、持续更新的榜单是否能够推动向脑一致性模型的更快收敛？

主要发现

• 该项目引入了一个大规模、数据丰富的基准（NSD）用于脑-模型对齐。
• 公开榜单促进了模型与人脑数据之间的快速、透明比较。
• 该倡议旨在促成合作，既培养出更符合大脑的 AI，又加深对大脑的理解。