[Paper Review] On the Detectability of ChatGPT Content: Benchmarking, Methodology, and Evaluation through the Lens of Academic Writing

Zeyan Liu, Zijun Yao+2 more|arXiv (Cornell University)|Jun 7, 2023

Artificial Intelligence in Healthcare and Education16 citations

TL;DR

This paper introduces GPABenchmark, a 600,000-sample dataset of human and GPT-generated academic abstracts, and proposes CheckGPT, a model-agnostic detector achieving ~98-99% accuracy on discipline-specific tasks and strong transfer to new domains.

ABSTRACT

With ChatGPT under the spotlight, utilizing large language models (LLMs) to assist academic writing has drawn a significant amount of debate in the community. In this paper, we aim to present a comprehensive study of the detectability of ChatGPT-generated content within the academic literature, particularly focusing on the abstracts of scientific papers, to offer holistic support for the future development of LLM applications and policies in academia. Specifically, we first present GPABench2, a benchmarking dataset of over 2.8 million comparative samples of human-written, GPT-written, GPT-completed, and GPT-polished abstracts of scientific writing in computer science, physics, and humanities and social sciences. Second, we explore the methodology for detecting ChatGPT content. We start by examining the unsatisfactory performance of existing ChatGPT detecting tools and the challenges faced by human evaluators (including more than 240 researchers or students). We then test the hand-crafted linguistic features models as a baseline and develop a deep neural framework named CheckGPT to better capture the subtle and deep semantic and linguistic patterns in ChatGPT written literature. Last, we conduct comprehensive experiments to validate the proposed CheckGPT framework in each benchmarking task over different disciplines. To evaluate the detectability of ChatGPT content, we conduct extensive experiments on the transferability, prompt engineering, and robustness of CheckGPT.

Motivation & Objective

• Motivate and quantify the challenges of detecting LLM-generated academic writing.
• Provide a comprehensive, cross-disciplinary dataset (GPABenchmark) for benchmarking detectors.
• Develop a model-agnostic detector (CheckGPT) that is accurate, transferable, and interpretable.

Proposed method

• Construct GPABenchmark with 600,000 samples spanning human-written, GPT-written, GPT-completed, and GPT-polished abstracts across CS, physics, and HSS.
• Evaluate existing open-source and commercial detectors (GPTZero, ZeroGPT, OpenAI’s classifier) on GPABenchmark across three tasks (GPT-WRI, GPT-CPL, GPT-POL).
• Design CheckGPT as a language-model-based detector with a general representation module and an attentive-BiLSTM classifier.
• Show CheckGPT is model-agnostic and transferable, requiring minimal domain-specific tuning.
• Conduct a human user study (150+ participants) to assess detectability of GPT-generated abstracts.

Figure 1: Distribution of abstract lengths (# of words): red: ChatGPT-generated; green: human-written.

Experimental results

Research questions

• RQ1 How well can humans distinguish GPT-generated abstracts across disciplines?
• RQ2 How do state-of-the-art detectors perform on GPABenchmark, especially for GPT-polished text?
• RQ3 Can a language-model-based detector (CheckGPT) achieve high accuracy and transferability without white-box access?
• RQ4 What insights into the generation process (explanability) can CheckGPT provide about LLM outputs?

Key findings

• GPABenchmark contains 600,000 samples with human-written, GPT-written, GPT-completed, and GPT-polished abstracts across CS, physics, and HSS.
• Human evaluators struggle to identify GPT-generated abstracts; accuracy is around random to modest, even among experts.
• Open-source and commercial detectors show unsatisfactory performance on GPABenchmark, particularly for GPT-polished text.
• CheckGPT achieves an average accuracy of 98%–99% on task-specific detectors and about 90% transfer accuracy to new domains without tuning, rising to ~98% with ~2,000 domain-tuned samples.
• CheckGPT is model-agnostic, lightweight, transferable, and provides interpretability insights into LLM-generated text.

Figure 2: The architecture of the CheckGPT model.