[Paper Review] Data Contamination Through the Lens of Time

Manley Roberts, Himanshu Thakur+3 more|arXiv (Cornell University)|Oct 16, 2023

Topic ModelingComputer Science3 citations

TL;DR

This paper presents the first large-scale longitudinal analysis of data contamination in large language models (LLMs) by leveraging known training cutoff dates of GPT-4 and GPT-3.5-Turbo. Using Codeforces and Project Euler—longitudinal coding benchmarks—it demonstrates statistically significant correlations between GitHub popularity and model pass rates only for pre-cutoff problems, providing strong empirical evidence of data contamination. The authors open-source their dataset and evaluation framework to enable rigorous, reproducible contamination analysis in future benchmarking.

ABSTRACT

Recent claims about the impressive abilities of large language models (LLMs) are often supported by evaluating publicly available benchmarks. Since LLMs train on wide swaths of the internet, this practice raises concerns of data contamination, i.e., evaluating on examples that are explicitly or implicitly included in the training data. Data contamination remains notoriously challenging to measure and mitigate, even with partial attempts like controlled experimentation of training data, canary strings, or embedding similarities. In this work, we conduct the first thorough longitudinal analysis of data contamination in LLMs by using the natural experiment of training cutoffs in GPT models to look at benchmarks released over time. Specifically, we consider two code/mathematical problem-solving datasets, Codeforces and Project Euler, and find statistically significant trends among LLM pass rate vs. GitHub popularity and release date that provide strong evidence of contamination. By open-sourcing our dataset, raw results, and evaluation framework, our work paves the way for rigorous analyses of data contamination in modern models. We conclude with a discussion of best practices and future steps for publicly releasing benchmarks in the age of LLMs that train on webscale data.

Motivation & Objective

• To investigate whether data contamination affects LLM performance on longitudinal coding benchmarks like Codeforces and Project Euler.
• To measure the extent of contamination by exploiting known training cutoff dates of GPT-4 and GPT-3.5-Turbo as natural experimental splits.
• To provide empirical evidence of contamination through statistical analysis of model performance trends across time-ordered problem releases.
• To promote best practices in public benchmark release by identifying risks posed by webscale LLM training data.
• To enable reproducible contamination analysis by open-sourcing datasets, evaluation code, and results.

Proposed method

• Uses known training cutoff dates of GPT-4 and GPT-3.5-Turbo to partition benchmark problems into pre-cutoff (likely seen) and post-cutoff (likely unseen) groups.
• Constructs longitudinal datasets from Codeforces (2010–2023) and Project Euler (2001–2023), tracking problem release dates and GitHub presence.
• Employs regression models to analyze the relationship between problem characteristics (e.g., GitHub popularity, difficulty) and LLM performance metrics (pass rate, title/tag reproduction).
• Measures model performance using test-case pass rates and semantic reproduction of titles and tags to detect memorization and contamination effects.
• Applies statistical inference to detect significant differences in performance trends between pre- and post-cutoff periods.
• Open-sources the full evaluation framework, raw results, and dataset construction pipeline under dataset licensing constraints.

Experimental results

Research questions

• RQ1 Is there a statistically significant performance difference in LLMs when evaluating on problems released before versus after their training cutoff?
• RQ2 Does the presence of a problem on GitHub correlate with higher pass rates in LLMs only for pre-cutoff problems, indicating contamination?
• RQ3 To what extent do LLMs reproduce problem titles and tags more accurately for problems released before their training cutoff?
• RQ4 How does model performance vary with problem difficulty and release date, and does this vary by model (GPT-4 vs. GPT-3.5-Turbo)?
• RQ5 What are the implications of contamination for the validity of public benchmarks in the era of webscale LLM training?

Key findings

• GPT-4 shows a statistically significant positive association (p = 0.010) between GitHub presence and pass rate only for pre-cutoff Codeforces problems, indicating contamination.
• For GPT-4, the coefficient for GitHub presence is 1.010 (95% CI: [0.994, 1.026]) in the pre-cutoff period, suggesting a slight but significant contamination effect.
• GPT-3.5-Turbo shows no significant association between GitHub presence and pass rate in either pre- or post-cutoff periods (p = 0.506 and p = 0.216 respectively).
• On the Project Euler dataset, GPT-4 shows a significant negative association (p = 0.00001) between difficulty and title reproduction in the pre-cutoff period, indicating possible memorization or contamination effects.
• For both models, the marginal effect of GitHub presence on title reproduction is not significant in post-cutoff periods, suggesting contamination is not present or detectable post-cutoff.
• The study finds no evidence of contamination in GPT-3.5-Turbo on Project Euler, as the GitHub presence coefficient is 0.981 (p = 0.696) in the pre-cutoff period.