[论文解读] Will ChatGPT get you caught? Rethinking of Plagiarism Detection

Mohammad Khalil, Erkan Er|arXiv (Cornell University)|Feb 8, 2023

Artificial Intelligence in Healthcare and Education被引用 47

一句话总结

这篇论文在50个主题上使用两个抄袭检测工具测试由 ChatGPT 生成的作文，发现原创性高且检测有限，并显示 ChatGPT 比传统工具更能识别自身输出。

ABSTRACT

The rise of Artificial Intelligence (AI) technology and its impact on education has been a topic of growing concern in recent years. The new generation AI systems such as chatbots have become more accessible on the Internet and stronger in terms of capabilities. The use of chatbots, particularly ChatGPT, for generating academic essays at schools and colleges has sparked fears among scholars. This study aims to explore the originality of contents produced by one of the most popular AI chatbots, ChatGPT. To this end, two popular plagiarism detection tools were used to evaluate the originality of 50 essays generated by ChatGPT on various topics. Our results manifest that ChatGPT has a great potential to generate sophisticated text outputs without being well caught by the plagiarism check software. In other words, ChatGPT can create content on many topics with high originality as if they were written by someone. These findings align with the recent concerns about students using chatbots for an easy shortcut to success with minimal or no effort. Moreover, ChatGPT was asked to verify if the essays were generated by itself, as an additional measure of plagiarism check, and it showed superior performance compared to the traditional plagiarism-detection tools. The paper discusses the need for institutions to consider appropriate measures to mitigate potential plagiarism issues and advise on the ongoing debate surrounding the impact of AI technology on education. Further implications are discussed in the paper.

研究动机与目标

• 在教育场景中评估 ChatGPT 生成的作文的原创性。
• 评估流行抄袭检测工具对 AI 生成内容的有效性。
• 考察 ChatGPT 是否能自我验证其生成的内容，并将其与传统检测工具进行比较。
• 讨论在教育中减缓 AI 辅助抄袭的制度性措施。

提出的方法

• 使用 ChatGPT 生成50篇关于不同主题的作文。
• 使用两种流行的抄袭检测工具评估输出的原创性。
• 请 ChatGPT 验证每篇作文是否由其自身生成，并将结果与检测工具的输出进行比较。
• 分析结果以讨论对教育和政策的影响。

实验结果

研究问题

• RQ1 抄袭检测工具是否能够可靠地识别出来自 ChatGPT 的 AI 生成的作文？
• RQ2 常见抄袭检测工具在 AI 生成内容与人类撰写内容相比的有效性如何？
• RQ3 ChatGPT 是否能够准确判断一篇作文是否由自身生成，以及这与检测工具的方法有何差异？

主要发现

• 根据所使用的检测工具，ChatGPT 能在许多主题上产出具有高原创性的文本。
• 两种流行的抄袭检测工具可能难以标记 ChatGPT 生成的内容。
• ChatGPT 的自我验证性能可能超过传统的抄袭检测工具。
• 研究结果支持对教育中易被AI辅助快捷方式的担忧，并强调需要制度层面的缓解措施。
• 本文讨论了关于教育中人工智能的持续辩论和政策考量。