[Paper Review] Cognitive Constraint Simulation and the Geometry of Human Authorship: A First-Principles Theory of AI Text Detection

Eric Mitchell|arXiv (Cornell University)|Jan 26, 2023

Topic Modeling152 citations

TL;DR

DetectGPT introduces a zero-shot text detection method that uses local curvature of a model’s log-probability to distinguish model-generated text from human-written text, without training on real/generated samples.

ABSTRACT

The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.

Motivation & Objective

• Identify whether a passage was generated by a specific language model using zero-shot detection.
• Explore whether machine-generated text occupies negative curvature regions of the model's log-probability function.
• Develop a practical detector that does not require labeled real/generated data or text watermarking.
• Validate the method across multiple datasets and model sizes and compare to existing zero-shot baselines.

Proposed method

• Hypothesize that model-generated text lies in regions of negative curvature of log p(x).
• Define perturbation discrepancy d(x,p,q) = log p(x) - E_{ ilde{x}~q(.|x)} log p( ilde{x}).
• Estimate -trace of Hessian of log p(x) via Hutchinson’s trace estimator and finite differences to approximate curvature.
• Use a mask-filling perturbation model (e.g., T5) to generate nearby passages on the data manifold.
• Compute normalized perturbation discrepancy and threshold to decide if x was generated by p.
• Provide algorithm DetectGPT that scores x using the source model p_theta and perturbations from q, with a threshold epsilon.

Figure 1: We aim to determine whether a piece of text was generated by a particular LLM $p$ , such as GPT-3. To classify a candidate passage $x$ , DetectGPT first generates minor perturbations of the passage $\tilde{x}_{i}$ using a generic pre-trained model such as T5. Then DetectGPT compares the lo

Experimental results

Research questions

• RQ1 Does text generated by a given LLM occupy more negative curvature regions of the model's log-probability than human-written text?
• RQ2 Can a zero-shot detector using local curvature information outperform existing zero-shot methods that rely on global log probabilities?
• RQ3 How robust is DetectGPT to paraphrasing, different perturbation models, and cross-model scoring scenarios?
• RQ4 What are the effects of model/perturbation capacity, perturbation quantity, and data domain on detection performance?

Key findings

• DetectGPT consistently improves AUROC over zero-shot baselines across multiple models (1.5B to 20B parameters) and domains.
• Perturbation discrepancy, linked to negative curvature, reliably separates model-generated from human-written text in empirical tests.
• Using larger mask-filled perturbation models (e.g., T5 variants) and more perturbations improves detection up to convergence around 100 perturbations.
• Detection remains strong under paraphrase/revision of machine-generated text and under different decoding strategies, with zero-shot detectors generally more domain-general than supervised ones.
• Cross-model scoring shows DetectGPT is most effective when scored by the same model that generated the text, but still offers utility with surrogate scorers.
• DetectGPT achieves AUROC improvements of around 0.1 over the strongest zero-shot baseline in several settings (as reported in the paper).

Figure 2: We identify and exploit the tendency of machine-generated passages $x\sim p_{\theta}(\cdot)$ (left) to lie in negative curvature regions of $\log p(x)$ , where nearby samples have lower model log probability on average. In contrast, human-written text $x\sim p_{real}(\cdot)$ (right) tends