[Paper Review] Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

Shahriar Golchin, Mihai Surdeanu|arXiv (Cornell University)|Nov 10, 2023

Topic Modeling4 citations

TL;DR

The paper introduces the Data Contamination Quiz (DCQ), a novel black-box method that detects and estimates data contamination in large language models (LLMs) by framing contamination detection as a multiple-choice reading comprehension task. By generating three word-level perturbed variants of dataset instances and including a 'none' option, DCQ leverages LLMs' memorization to identify exposure to training data—achieving contamination estimates up to 80x higher than prior state-of-the-art methods without access to model weights or pre-training data.

ABSTRACT

We propose the Data Contamination Quiz (DCQ), a simple and effective approach to detect data contamination in large language models (LLMs) and estimate the amount of it. Specifically, we frame data contamination detection as a series of multiple-choice questions, devising a quiz format wherein three perturbed versions of each instance, subsampled from a specific dataset partition, are created. These changes only include word-level perturbations. The generated perturbations, along with the original dataset instance, form the options in the DCQ, with an extra option accommodating the selection of none of the provided options. Given that the only distinguishing signal among the options is the exact wording with respect to the original dataset instance, an LLM, when tasked with identifying the original dataset instance, gravitates towards selecting the original one if it has been exposed to it. While accounting for positional biases in LLMs, the quiz performance reveals the contamination level for the tested model with the dataset partition to which the quiz pertains. Applied to various datasets and LLMs, under controlled and uncontrolled contamination, our findings, while fully lacking access to training data and model parameters, suggest that DCQ achieves state-of-the-art results and uncovers greater contamination levels through memorization compared to existing methods. Also, it proficiently bypasses more safety filters, especially those set to avoid generating copyrighted content.

Motivation & Objective

• To address the lack of effective, scalable methods for detecting data contamination in black-box LLMs without access to pre-training data or model weights.
• To develop a technique that quantifies contamination levels rather than merely detecting its presence.
• To overcome limitations of existing methods, including susceptibility to safety filters and overestimation due to positional biases in LLMs.
• To create a practical, low-resource solution for evaluating LLM robustness and fairness in real-world deployment.

Proposed method

• Formulate a five-option multiple-choice quiz per dataset instance: one original, three word-level perturbed versions using contextually relevant synonyms, and a 'none' option.
• Generate perturbed variants using zero-shot prompting with GPT-4, ensuring semantic similarity while altering lexical form.
• Measure LLM performance on identifying the original instance as an indicator of memorization and data contamination.
• Systematically permute the position of the original instance across quiz options to mitigate positional bias and report maximum performance as contamination estimate.
• Use the highest accuracy across multiple permutations as the robust contamination estimate, accounting for LLMs’ positional preferences.
• Validate results via reciprocal evaluation with prior replication-based methods, confirming consistency and higher detection sensitivity.

Experimental results

Research questions

• RQ1 Can a multiple-choice reading comprehension task effectively detect data contamination in black-box LLMs without access to model weights or pre-training data?
• RQ2 How does the DCQ method compare to state-of-the-art contamination detection techniques in terms of sensitivity and estimation accuracy?
• RQ3 To what extent can DCQ bypass safety filters designed to prevent generation of copyrighted or memorized content?
• RQ4 Does the method remain robust against positional biases inherent in LLMs, and how is this mitigated in practice?
• RQ5 Can DCQ detect contamination in real-world, uncontrolled datasets where ground-truth contamination levels are unknown?

Key findings

• DCQ detected contamination levels up to 80 times higher than the state-of-the-art method proposed by Golchin & Surdeanu (2023), even without access to pre-training data.
• In controlled settings, DCQ’s contamination estimates closely aligned with OpenAI’s reported figures but revealed significantly higher contamination levels, especially in high-impact benchmarks like GSM8k.
• For real-world datasets such as WNLI and XSum, DCQ detected contamination rates of up to 50.70% and 95.00%, respectively—demonstrating detection capability across diverse domains.
• The method successfully bypassed safety filters designed to block copyrighted content generation, indicating resilience to common mitigation strategies.
• Reciprocal evaluation with replication-based methods showed perfect agreement, confirming DCQ’s reliability and higher sensitivity in detecting memorization.
• DCQ outperformed both extraction-based and replication-based approaches in contamination estimation, particularly in scenarios with low to moderate contamination where other methods failed to detect any.