[论文解读] Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sébastien Bubeck, Varun Chandrasekaran+12 more|arXiv (Cornell University)|Mar 22, 2023

Artificial Intelligence in Healthcare and Education被引用 1,528

一句话总结

这篇论文提出了对 GPT-4 的早期研究，认为它在语言、数学、编码、视觉、医学、法律等领域展现出广泛的人类水平能力，暗示它是走向通用人工智能的一步，同时指出了局限性和社会影响。

ABSTRACT

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

研究动机与目标

• 证明 GPT-4 超越语言本身的广泛跨领域能力。
• 评估 GPT-4 是否展现出接近人类表现的通用智能或涌现行为。
• 调查 GPT-4 的局限性、失效模式和偏见，以概述通向 AGI 路径上的挑战。
• 讨论潜在通用 AI 跃升的社会影响与治理考量。

提出的方法

• 使用自然语言提示在多样领域（语言、数学、编码、视觉、医学、法律、心理学）与早期 GPT-4 实例互动。
• 将 GPT-4 输出与早前模型（如 ChatGPT）进行比较，以评估通用性和性能差距。
• 引出目标任务（如多模态推理、工具使用、计划制定），以探查超越记忆的通用能力。
• 变化提示以测试适应性、风格灵活性和解决问题的方法。
• 记录局限性、偏见和失败模式，以识别通往更深层次 AGI 能力的障碍。

实验结果

研究问题

• RQ1 Does GPT-4 demonstrate general, cross-domain abilities beyond language tasks?
• RQ2 To what extent does GPT-4 approach human-level performance across diverse domains without task-specific prompting?
• RQ3 What are the primary limitations, failure modes, and biases that constrain GPT-4’s general intelligence?
• RQ4 What societal and ethical implications accompany a system exhibiting broad, AGI-like capabilities?

主要发现

• GPT-4 exhibits capabilities across mathematics, coding, vision, medicine, law, and psychology in addition to language.
• GPT-4’s performance in many tasks is close to human-level and often surpasses prior models like ChatGPT.
• GPT-4 demonstrates emergent, non-human-like patterns of intelligence and adaptability across domains.
• The model shows limitations in planning, arithmetic, and some reasoning tasks, highlighting gaps toward full AGI.
• There are notable concerns about misinformation, bias, and societal impact that accompany advanced LLM capabilities.