[논문 리뷰] Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sébastien Bubeck, Varun Chandrasekaran외 12명|arXiv (Cornell University)|2023. 03. 22.

Artificial Intelligence in Healthcare and Education인용 수 1,528

한 줄 요약

본 논문은 GPT-4의 초기 연구를 제시하며, 언어, 수학, 코딩, 비전, 의학, 법률 등 다양한 영역에서 광범위하고 인간 수준의 능력을 보인다고 주장하고, AGI로의 한 걸음임을 시사하는 한편 한계와 사회적 함의에 주목한다.

ABSTRACT

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

연구 동기 및 목표

• GPT-4의 광범위한 교차 도메인 능력을 언어 그 자체를 넘어 입증한다.
• GPT-4가 일반 지능이나 인간 성능에 근접한 발현 행동을 보이는지 평가한다.
• GPT-4의 한계, 실패 모드 및 편향을 조사하여 AGI 경로에서의 도전과제를 제시한다.
• 잠재적 일반 AI 도약의 사회적 영향 및 거버넌스 고려사항을 논의한다.

제안 방법

• 다양한 도메인(언어, 수학, 코딩, 비전, 의학, 법률, 심리학)에 걸친 자연어 프롬프트를 사용하여 초기 GPT-4 인스턴스와 상호 작용한다.
• 일반성 및 성능 격차를 평가하기 위해 GPT-4 출력과 이전 모델들(예: ChatGPT)을 비교한다.
• 다중 모달 추론, 도구 사용, 계획 수립 등 표적 작업을 이끌어내어 암기화를 넘어서는 범용 기능을 탐구한다.
• 적응성, 스타일 유연성 및 문제 해결 방식 테스트를 위해 프롬프트를 다양화한다.
• 깊은 AGI 기능으로의 장애물을 식별하기 위해 한계, 편향 및 실패 모드를 문서화한다.

실험 결과

연구 질문

• RQ1 Does GPT-4 demonstrate general, cross-domain abilities beyond language tasks?
• RQ2 To what extent does GPT-4 approach human-level performance across diverse domains without task-specific prompting?
• RQ3 What are the primary limitations, failure modes, and biases that constrain GPT-4’s general intelligence?
• RQ4 What societal and ethical implications accompany a system exhibiting broad, AGI-like capabilities?

주요 결과

• GPT-4 exhibits capabilities across mathematics, coding, vision, medicine, law, and psychology in addition to language.
• GPT-4’s performance in many tasks is close to human-level and often surpasses prior models like ChatGPT.
• GPT-4 demonstrates emergent, non-human-like patterns of intelligence and adaptability across domains.
• The model shows limitations in planning, arithmetic, and some reasoning tasks, highlighting gaps toward full AGI.
• There are notable concerns about misinformation, bias, and societal impact that accompany advanced LLM capabilities.