[论文解读] A Categorical Archive of ChatGPT Failures

Ali Borji|arXiv (Cornell University)|Feb 6, 2023

Artificial Intelligence in Healthcare and Education被引用 91

一句话总结

这篇论文通过将 ChatGPT 的失败分成十一类来分析，展示在推理、数学、编码、偏见、事实准确性等方面的局限性，以指导未来的改进。

ABSTRACT

Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study. Eleven categories of failures, including reasoning, factual errors, math, coding, and bias, are presented and discussed. The risks, limitations, and societal implications of ChatGPT are also highlighted. The goal of this study is to assist researchers and developers in enhancing future language models and chatbots.

研究动机与目标

• 识别并对 ChatGPT 常见失败模式进行分类，建立一个用于评估随时间进展的参考。
• 突出大型语言模型的风险、局限性及社会影响，以引导负责任的发展。
• 提供类似数据集的失败示例参考，以帮助模型训练和测试。

提出的方法

• 汇编来自公开演示和先前工作的失败示例。
• 将失败分为十一类，包括推理、逻辑、数学、事实错误、偏见、幽默和编码。
• 讨论每一类别及其代表性实例及相关含义。

实验结果

研究问题

• RQ1 哪些类别最能涵盖 ChatGPT 在多样任务中的典型失败模式？
• RQ2 这些失败在推理、算术、事实准确性和社会偏见方面如何表现？
• RQ3 这些失败对安全、伦理和未来模型发展有哪些影响？

主要发现

• eleven 类别的 ChatGPT 失败被识别并讨论。
• Failures span reasoning, logic, math and arithmetic, factual errors, bias and discrimination, wit and humor, coding, and more.
• The analysis emphasizes limitations like lack of a world model, susceptibility to hallucinations, and the need for standardized benchmarks.
• Examples show improvements over time in some areas (e.g., physical reasoning), but many challenges persist across categories.
• The work proposes the archive as a reference point for evaluating model progress and for generating synthetic data for training and testing.