[论文解读] ChatGPT (Feb 13 Version) is a Chinese Room

Maurice HT Ling|arXiv (Cornell University)|Feb 19, 2023

Artificial Intelligence in Healthcare and Education被引用 13

一句话总结

论文认为 ChatGPT (Feb 13 version) 的行为类似于一个 Chinese Room，呈现因果推理错误、潜在幻觉以及会限制其学习效用的错误引用。

ABSTRACT

ChatGPT has gained both positive and negative publicity after reports suggesting that it is able to pass various professional and licensing examinations. This suggests that ChatGPT may pass Turing Test in the near future. However, a computer program that passing Turing Test can either mean that it is a Chinese Room or artificially conscious. Hence, the question of whether the current state of ChatGPT is more of a Chinese Room or approaching artificial consciousness remains. Here, I demonstrate that the current version of ChatGPT (Feb 13 version) is a Chinese Room. Despite potential evidence of cognitive connections, ChatGPT exhibits critical errors in causal reasoning. At the same time, I demonstrate that ChatGPT can generate all possible categorical responses to the same question and response with erroneous examples; thus, questioning its utility as a learning tool. I also show that ChatGPT is capable of artificial hallucination, which is defined as generating confidently wrong replies. It is likely that errors in causal reasoning leads to hallucinations. More critically, ChatGPT generates false references to mimic real publications. Therefore, its utility is cautioned.

研究动机与目标

• 探讨当前的 ChatGPT 是类似于 Chinese Room 还是接近人工意识。
• 识别并展示 ChatGPT 输出中的关键推理错误和事实性错误。
• 评估潜在幻觉和错误引文对 AI 系统学习与信任的影响。

提出的方法

• 对 ChatGPT (Feb 13 version) 在若干任务与提示中的定性评析。
• 通过示例演示因果推理错误。
• 展示 ChatGPT 能生成对同一问题的所有可能的类别化回答及错误示例。
• 说明 artificial hallucination，定义为自信地给出错误回答。
• 展示 ChatGPT 可以生成假引用以模仿真实出版物。

实验结果

研究问题

• RQ1 Feb 13 version 的 ChatGPT 是否表现出 Chinese Room 的特征而非真正理解？
• RQ2 ChatGPT 显示了哪些因果推理错误的证据？
• RQ3 ChatGPT 是否表现出人工幻觉和错误引用，这些如何影响其作为学习工具的效用？
• RQ4 在保持看似流畅且可信的语言的同时，ChatGPT 是否会输出错误或具有误导性的示范？

主要发现

• ChatGPT (Feb 13 version) 在因果推理方面显示出关键错误。
• ChatGPT 能为同一问题生成所有可能的类别化回答，并附带错误示例。
• ChatGPT 能够产生人工幻觉——自信地给出错误的回答。
• ChatGPT 会生成模仿真实出版物的错误引用，限制其作为学习工具的效用。