[Paper Review] Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore

Junchao Wu, Runzhe Zhan+5 more|arXiv (Cornell University)|May 7, 2024

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes GECScore, a black-box zero-shot detector for LLM-generated text that leverages grammar error correction to distinguish between human-written and model-generated text. By computing the similarity between original and grammatically corrected versions using a pre-trained grammar correction model, GECScore achieves 98.7% average AUROC and demonstrates strong robustness against paraphrasing and adversarial attacks.

ABSTRACT

The efficacy of detectors for texts generated by large language models (LLMs) substantially depends on the availability of large-scale training data. However, white-box zero-shot detectors, which require no such data, are limited by the accessibility of the source model of the LLM-generated text. In this paper, we propose a simple yet effective black-box zero-shot detection approach based on the observation that, from the perspective of LLMs, human-written texts typically contain more grammatical errors than LLM-generated texts. This approach involves calculating the Grammar Error Correction Score (GECScore) for the given text to differentiate between human-written and LLM-generated text. Experimental results show that our method outperforms current state-of-the-art (SOTA) zero-shot and supervised methods, achieving an average AUROC of 98.62% across XSum and Writing Prompts dataset. Additionally, our approach demonstrates strong reliability in the wild, exhibiting robust generalization and resistance to paraphrasing attacks. Data and code are available at: https://github.com/NLP2CT/GECScore.

Motivation & Objective

• Address the limitations of existing zero-shot detectors that rely on access to the source LLM or are vulnerable to paraphrased or adversarial inputs.
• Overcome the dependency on large-scale training data or fine-tuned classifiers in supervised methods, especially in out-of-distribution settings.
• Develop a black-box detection approach independent of the source model, enabling deployment in real-world scenarios with unknown or black-box LLMs.
• Leverage the cognitive insight that human-written text contains more grammatical errors than LLM-generated text to design a robust detection signal.
• Achieve state-of-the-art performance in zero-shot detection without requiring model-specific access or extensive data collection.

Proposed method

• Propose GECScore, a detection framework that computes the similarity between an input text and its grammatically corrected version using a pre-trained grammar error correction (GEC) model.
• Apply a similarity metric (e.g., BLEU, TER, chrF, Edit Distance, ROUGE, METEOR, BLEURT) to measure the textual similarity between the original and corrected texts.
• Use a softmax-scaled similarity score to amplify the distinction between human-written and LLM-generated texts, where higher similarity indicates higher likelihood of LLM generation.
• Set a threshold ε on the softmax-scaled score; if the score exceeds ε, classify the text as LLM-generated.
• Operate in a black-box setting—no access to the LLM’s logits, weights, or training data is required—making it universally applicable.
• Utilize the Working Memory Theory and cognitive psychology insights to justify the hypothesis that human-written text has higher grammatical error rates than LLM outputs.

Figure 1: GECScore Framework Overview. First, a grammar correction model generates a grammatically corrected version $\tilde{x_{i}}$ of the input text $x_{i}$ . Next, the similarity score $\hat{s}_{x_{i}}$ between $\tilde{x_{i}}$ and $x_{i}$ is calculated using a similarity metric $Sim$ , and then a

Experimental results

Research questions

• RQ1 Can a black-box zero-shot detector be designed that does not require access to the source LLM or large-scale training data?
• RQ2 Does the inherent grammatical accuracy of LLM-generated text compared to human-written text provide a reliable signal for zero-shot detection?
• RQ3 Can a grammar correction-based similarity score effectively distinguish between human and LLM-generated text across diverse text types and distributions?
• RQ4 How robust is the proposed method against paraphrasing and adversarial perturbations that evade existing detectors?
• RQ5 Can this approach outperform both state-of-the-art zero-shot and supervised detection methods in terms of AUROC and generalization?

Key findings

• GECScore achieves an average AUROC of 98.7% across multiple benchmarks, outperforming current state-of-the-art zero-shot and supervised methods.
• The method demonstrates strong robustness against paraphrase attacks, maintaining high detection accuracy even when input texts are rephrased.
• GECScore remains effective against adversarial perturbations, showing resilience where other detectors fail.
• The approach is entirely black-box: it does not require access to the LLM’s weights, logits, or fine-tuned classifiers.
• The method is computationally efficient and scalable, relying only on a pre-trained GEC model and standard text similarity metrics.
• The superiority of GECScore is attributed to its reliance on a fundamental linguistic signal—grammatical error frequency—rooted in cognitive psychology and working memory constraints.

Figure 2: Similarity Score Distribution for Grammatically Corrected versus Original Texts. This figure illustrates the distribution of similarity scores using the BLEURT metric for 500 human-written and GPT-3.5 generated samples from the Writing Prompts dataset. Post-softmax scaling, a marked diverg