[Paper Review] People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text

Jenna Russell, Marzena Karpinska+1 more|ArXiv.org|Jan 26, 2025

Artificial Intelligence in Healthcare and Education3 citations

TL;DR

Human annotators who frequently use LLMs for writing tasks detect AI-generated text with high accuracy, outperforming most automatic detectors, even under paraphrasing and humanization tricks. A five-expert majority vote achieves near-perfect accuracy on 300 articles.

ABSTRACT

In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1). We hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such "expert" annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. Qualitative analysis of the experts' free-form explanations shows that while they rely heavily on specific lexical clues ('AI vocabulary'), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity) that are challenging to assess for automatic detectors. We release our annotated dataset and code to spur future research into both human and automated detection of AI-generated text.

Motivation & Objective

• Identify whether frequent LLM users can reliably detect AI-generated non-fiction text without training.
• Compare expert human detectors to automatic detectors across modern LLMs (GPT-4o, Claude-3.5-Sonnet, o1-Pro).
• Analyze features used by experts to distinguish AI from human writing.
• Assess robustness of expert detection under evasion tactics (paraphrasing, humanization).
• Provide annotated data and code to spur future work in AI-generated text detection.

Proposed method

• Recruit human annotators to label 300 non-fiction articles as human- or AI-written, with paragraph-length explanations.
• Identify a subset of five annotators who frequently use LLMs for writing tasks as “experts.”
• Have experts and non-experts judge batches of articles generated by multiple LLMs, including paraphrased and humanized variants.
• Evaluate detectors via true positive rate (TPR) and false positive rate (FPR), calibrating thresholds to fixed FPRs where needed.
• Compare expert majority vote to automatic detectors (Pangram, GPTZero, Binoculars, Fast-DetectGPT) and prompt-based detectors.
• Release annotated dataset and code for future research.

Figure 1: A human expert’s annotations of an article generated by OpenAI’s o1-Pro with humanization. The expert provides a judgment on whether the text is written by a human or AI, a confidence score, and an explanation (including both free-form text and highlighted spans) of their decision.

Experimental results

Research questions

• RQ1 Can a population of humans who frequently use LLMs for writing reliably detect AI-generated text without training?
• RQ2 Do expert human detectors generalize across different LLMs beyond GPT-4o?
• RQ3 How robust are expert detectors to paraphrasing and humanization attacks?
• RQ4 How do expert detectors compare to state-of-the-art automatic detectors on a controlled test set?
• RQ5 What cues do experts rely on when differentiating AI from human writing?

Key findings

• Expert annotators who frequently use LLMs achieve high detection accuracy (TPR ≈ 92.7% with FPR ≈ 3.3% on initial batch; majority vote reaches near-perfect performance).
• The expert majority vote outperforms most automatic detectors across multiple models and evasion tactics, with Pangram matching their near-perfect accuracy in some configurations.
• Paraphrasing and humanization do not substantially degrade expert performance; experts remain robust to these evasion strategies.
• Experts rely on AI vocabulary, formulaic structures, and originality, but also consider tone, clarity, and factuality in judgments.
• Even new model paradigms (e.g., o1-Pro) can be detected reliably by the expert majority, though confidence may wane for particularly challenging instances.
• The dataset and code are released to support future research in human and automated AI-generated text detection.

Figure 2: Expert confidence in their decisions drops when judging humanized articles generated by o1-Pro .