[Paper Review] A non-parametric test to detect data-copying in generative models

Casey Meehan, Kamalika Chaudhuri+1 more|arXiv (Cornell University)|Apr 12, 2020

Artificial Intelligence in Games11 citations

TL;DR

This paper proposes a non-parametric three-sample test to detect data-copying in generative models—where models memorize and reproduce training samples or minor variations. Using the training set, a target distribution sample, and a generated sample, the test identifies memorization without parametric assumptions, demonstrating strong detection performance across multiple models and datasets.

ABSTRACT

Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} -- where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample non-parametric test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets. For code \& examples, visit this https URL

Motivation & Objective

• To formalize and define data-copying as a specific form of overfitting in generative models.
• To address the challenge of detecting memorization of training samples or their small variations.
• To develop a non-parametric, three-sample test that does not rely on model-specific assumptions.
• To evaluate the test's effectiveness across diverse canonical models and datasets.

Proposed method

• The test uses three inputs: the training set, a separate sample from the target distribution, and a sample generated by the model.
• It applies a non-parametric statistical test based on the relative ordering of distances between samples.
• The method compares distances from generated samples to training and target samples to detect anomalous similarity.
• It avoids parametric assumptions by relying on rank-based statistics and permutation testing.
• The test is designed to be model-agnostic and applicable to any generative model.
• The procedure is validated through empirical evaluation on multiple benchmark models and datasets.

Experimental results

Research questions

• RQ1 Can a non-parametric test reliably detect data-copying in generative models without assuming model structure?
• RQ2 How effective is the three-sample test in identifying memorized samples or their variations?
• RQ3 Does the test maintain high power across diverse models and datasets?
• RQ4 How does the test perform under varying levels of memorization and data distribution shifts?

Key findings

• The proposed test successfully detects data-copying in generative models across multiple benchmark datasets and architectures.
• The method achieves high detection power even when memorization is subtle or limited to small variations of training samples.
• The test remains robust across different model types, including GANs, VAEs, and autoregressive models.
• Empirical results show the test outperforms baseline methods in detecting memorization without requiring model-specific tuning.
• The test's performance is consistent across diverse data distributions, indicating generalizability.
• The approach enables reliable detection of memorization without access to model internals or parametric assumptions.