[论文解读] Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity

Shushanta Pudasaini, Luis Miralles‐Pechuán+2 more|arXiv (Cornell University)|Jun 4, 2024

Academic integrity and plagiarism被引用 9

一句话总结

本文综述了像 ChatGPT 与 Gemini 这样的大语言模型（LLMs）如何影响学术诚信，评估针对 AI 生成内容和抄袭的检测方法，并讨论存在的差距与未来的解决方案。

ABSTRACT

The rise of Large Language Models (LLMs) such as ChatGPT and Gemini has posed new challenges for the academic community. With the help of these models, students can easily complete their assignments and exams, while educators struggle to detect AI-generated content. This has led to a surge in academic misconduct, as students present work generated by LLMs as their own, without putting in the effort required for learning. As AI tools become more advanced and produce increasingly human-like text, detecting such content becomes more challenging. This development has significantly impacted the academic world, where many educators are finding it difficult to adapt their assessment methods to this challenge. This research first demonstrates how LLMs have increased academic dishonesty, and then reviews state-of-the-art solutions for academic plagiarism in detail. A survey of datasets, algorithms, tools, and evasion strategies for plagiarism detection has been conducted, focusing on how LLMs and AI-generated content (AIGC) detection have affected this area. The survey aims to identify the gaps in existing solutions. Lastly, potential long-term solutions are presented to address the issue of academic plagiarism using LLMs based on AI tools and educational approaches in an ever-changing world.

研究动机与目标

• 展示 LLMs 如何增加学术不端行为及其对抄袭检测的影响。
• 综述 AI 生成内容检测的前沿数据集、算法、工具以及规避策略。
• 识别当前检测器在缺口、局限性和评估挑战方面的问题。
• 讨论从長期技术与教育层面解决 AI 驱动抄袭的对策。

提出的方法

• 回顾现有关于抄袭与 AIGC 检测的文献。
• 编目用于 AIGC 检测的数据集、检测算法与工具。
• 检查规避技术及其对检测器可靠性的影响。
• 分析水印、零样本（zero-shot）及其他检测方法。
• 突出差距并提出潜在基准与教育性解决方案。

Figure 1: Timeline indicating the release date and parameter of different GPT models by OpenAI.

实验结果

研究问题

• RQ1 像 ChatGPT 与 Gemini 这样的 LLMs 如何影响学术不端行为与抄袭检测？
• RQ2 用于 AI 生成内容检测的主要数据集、算法与工具有哪些，它们的有效性如何？
• RQ3 存在哪些规避检测的策略，它们对现有解决方案有何影响？
• RQ4 当前 AIGC/抄袭检测方法存在哪些差距，提出了哪些未来方向？
• RQ5 从教育和技术角度，在长期内可以如何解决 AI 驱动的抄袭？

主要发现

• LLMs 已加剧学术不端行为并使传统抄袭检测变得更加复杂。
• 存在用于 AIGC 检测的广泛数据集、算法和工具，持续的规避策略对可靠性构成挑战。
• 水印、零样本（zero-shot）和基于提示的方法构成核心检测途径，各自在人健壮性与实用性方面存在权衡。
• 基准数据集和领域多样性方面存在显著差距，阻碍跨研究的公平对比。
• 作为对 AI 生成抄袭的整体性应对的一部分，讨论了若干教育与政策导向的解决方案。

Figure 2: Diagram demonstrating how ChatGPT and paraphrasing tools can be used to complete assignments.