[Paper Review] Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

Kalpesh Krishna, Yixiao Song+3 more|arXiv (Cornell University)|Mar 23, 2023

Topic ModelingComputer Science89 citations

TL;DR

The paper shows that a discourse-aware paraphraser (dipper) can evade AI-generated text detectors, while a retrieval-based defense using semantically matching prior generations can effectively detect paraphrased outputs.

ABSTRACT

The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated text, including those based on watermarking or outlier detection. However, the robustness of these detection algorithms to paraphrases of AI-generated text remains unclear. To stress test these detectors, we build a 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIPPER drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics. To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider. Given a candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within a certain threshold. We empirically verify our defense using a database of 15M generations from a fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% of human-written sequences as AI-generated. We open-source our models, code and data.

Motivation & Objective

• Assess robustness of AI-generated text detectors to paraphrase attacks.
• Develop a discourse-level paraphraser capable of long-form text paraphrasing with controllable diversity.
• Evaluate detectors across multiple LMs and tasks under paraphrase attacks.
• Propose and evaluate a retrieval-based defense that detects AI-generated text via semantic matching to a database of prior generations.

Proposed method

• Train an 11B parameter paraphrase model (dipper) to paraphrase paragraph-length text with context and content reordering controls.
• Fine-tune dipper on paragraph-level paraphrase data with context and diversity controls (lexical diversity L and content order O).
• Evaluate detectors (watermarking, DetectGPT, GPTZero, OpenAI’s classifier, RankGen) on GPT2-XL, OPT-13B, and GPT-3.5-davinci-003 with paraphrased outputs.
• Measure semantic preservation via P-SP semantic similarity to human paraphrases.
• Demonstrate a retrieval-based defense that stores all API outputs and detects paraphrase by matching candidate text to the database using BM25 or P-SP representations.
• Scale retrieval to large corpora (up to 15M generations) and analyze detection performance under paraphrase attacks.

Experimental results

Research questions

• RQ1 How robust are existing AI-generated text detectors to paraphrase attacks generated by a large paraphrase model?
• RQ2 Can a discourse-aware paraphrase model effectively evade detectors while preserving semantics?
• RQ3 Is retrieval over a large corpus of previously-generated sequences a robust defense against paraphrase attacks, and how does it scale?
• RQ4 What are the practical limitations and scalability considerations of retrieval-based detection for AI-generated text?
• RQ5 How do different detectors compare to retrieval-based methods under paraphrase attacks across multiple LMs and tasks?

Key findings

• Paraphrasing with dipper substantially lowers detector accuracy across detectors and models (e.g., DetectGPT from 70.3% to 4.6% at 1% FPR for GPT2-XL).
• Dipper achieves high semantic preservation, with 88%-99% of paraphrases exceeding the median human-paraphrase semantic similarity (P-SP).
• Retrieval-based detection achieves high detection rates on paraphrased text, outperforming baselines (e.g., 97.3%-97.8% with BM25 over 9K or 43K+ corpora for various models).
• BM25-based retrieval over a corpus of 15M generations remains effective as corpus size grows, with only modest drops in accuracy (e.g., from 98.3% to 97.3% on PG19 when scaling to 15M).
• Retrieval-based detection remains robust across query lengths, performing best with 50+ tokens per query.