[论文解读] From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks

Jacob Russin, Sam Whitman McGrath+1 more|arXiv (Cornell University)|May 24, 2024

Biomedical Text Mining and Ontologies被引用 4

一句话总结

本文研究了现代深度神经网络（DNNs），尤其是大语言模型（LLMs），如何在缺乏显式符号架构的情况下，实现组合泛化——即组合已知元素形成新颖且有意义的结构。文章认为，架构的归纳偏置和元学习（包括LLM预训练）使DNNs能够复制组合性的核心行为特征，从而挑战了长期以来认为唯有经典符号系统才能支持系统性、创造性的认知的观点。

ABSTRACT

Compositionality has long been considered a key explanatory property underlying human intelligence: arbitrary concepts can be composed into novel complex combinations, permitting the acquisition of an open ended, potentially infinite expressive capacity from finite learning experiences. Influential arguments have held that neural networks fail to explain this aspect of behavior, leading many to dismiss them as viable models of human cognition. Over the last decade, however, modern deep neural networks (DNNs), which share the same fundamental design principles as their predecessors, have come to dominate artificial intelligence, exhibiting the most advanced cognitive behaviors ever demonstrated in machines. In particular, large language models (LLMs), DNNs trained to predict the next word on a large corpus of text, have proven capable of sophisticated behaviors such as writing syntactically complex sentences without grammatical errors, producing cogent chains of reasoning, and even writing original computer programs -- all behaviors thought to require compositional processing. In this chapter, we survey recent empirical work from machine learning for a broad audience in philosophy, cognitive science, and neuroscience, situating recent breakthroughs within the broader context of philosophical arguments about compositionality. In particular, our review emphasizes two approaches to endowing neural networks with compositional generalization capabilities: (1) architectural inductive biases, and (2) metalearning, or learning to learn. We also present findings suggesting that LLM pretraining can be understood as a kind of metalearning, and can thereby equip DNNs with compositional generalization abilities in a similar way. We conclude by discussing the implications that these findings may have for the study of compositionality in human cognition and by suggesting avenues for future research.

研究动机与目标

• 评估现代深度神经网络（DNNs）是否真正能够表现出组合泛化，这是人类认知的标志性特征。
• 通过证明DNNs能够以非符号方式复制组合性行为，挑战经典观点，即组合性需要符号架构。
• 探讨架构的归纳偏置和元学习（包括大规模预训练）如何使DNNs实现组合泛化。
• 评估这些发现对认知科学的影响，特别是关于心理表征的本质以及组合性在解释中的作用。
• 倡导使用机制可解释性作为工具，以确定DNNs是否以在功能上具有意义且在理论上具有信息量的方式实现组合性。

提出的方法

• 综述机器学习中的实证研究，以识别使DNNs实现组合泛化的机制。
• 分析架构的归纳偏置（如变压器中的注意力机制）作为内置组合性归纳偏置的来源。
• 考察元学习，特别是大语言模型的大规模预训练，作为一种“学会学习”的形式，使DNNs能够泛化到未见过的组合结构。
• 通过展示元学习与经典组合性之间的类比，说明两者均能实现对已见样本之外的泛化。
• 应用机制可解释性技术，探究DNNs的内部表征是否反映组合结构。
• 参与关于实现与解释的哲学辩论，以评估DNNs是否仅在实现经典系统，还是提供了新颖的认知洞见。

Figure 1: Transformer basics. A) Comparison with recurrent neural networks (RNNs) in the language modeling setting (predicting the next word). A fundamental difference between RNNs and transformers is that RNNs must compress the entire past into a single hidden vector. To see this, notice how the on

实验结果

研究问题

• RQ1 深度神经网络是否能在没有显式符号架构的情况下实现组合泛化？
• RQ2 现代DNNs中的架构归纳偏置（如注意力机制）在多大程度上支持组合性行为？
• RQ3 元学习——特别是大语言模型的预训练——如何赋予DNNs组合泛化能力？
• RQ4 DNNs在复制组合性行为方面的成功，对组合性的哲学与认知科学地位意味着什么？
• RQ5 机制可解释性能否揭示DNNs是否以在功能上类似于人类认知的方式实现组合性？

主要发现

• 现代DNNs中的架构归纳偏置，尤其是变压器中的注意力机制，为组合泛化提供了结构基础。
• 元学习，特别是大语言模型在海量文本语料上的预训练，作为一种“学会学习”的形式，使DNNs能够泛化到新颖的组合结构。
• LLMs在句法复杂句子生成、推理链和程序合成等任务中表现出稳健的组合泛化能力。
• LLMs的预训练可被理解为一种元学习形式，使模型获得与显式元学习协议所导出的相似的组合泛化能力。
• 机制可解释性研究表明，DNNs可能以部分类比于经典理论但又在有意义且具信息量的方式产生差异的方式实现组合结构。
• 研究结果挑战了组合性仅属于符号系统的观点，表明非符号架构也能支持系统性、创造性的认知。

Figure 2: Are neural networks stimulus-response learners, or do they learn to solve problems with compositional algorithms? Two extreme possibilities are depicted for natural language (top) and simple arithmetic (bottom). In either case, a stimulus-response learner (left) would simply memorize a loo