[Paper Review] Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Models

Nick Dodson, Xinyu Gao+3 more|arXiv (Cornell University)|Feb 19, 2026

Privacy-Preserving Technologies in Data0 citations

TL;DR

The paper presents a geometric framework with three noise-regimes (small, medium, large) identifying a dangerous middle regime where memorization peaks, and proposes a geometry-driven mitigation by undertraining this regime while preserving generation quality.

ABSTRACT

Diffusion models generate high-quality samples but can also memorize training data, raising serious privacy concerns. Understanding the mechanisms governing when memorization versus generalization occurs remains an active area of research. In particular, it is unclear where along the noise schedule memorization is induced, how data geometry influences it, and how phenomena at different noise scales interact. We introduce a geometric framework that partitions the noise schedule into three regimes based on the coverage properties of training data by Gaussian shells and the concentration behavior of the posterior, which we argue are two fundamental objects governing memorization and generalization in diffusion models. This perspective reveals that memorization risk is highly non-uniform across noise levels. We further identify a danger zone at medium noise levels where memorization is most pronounced. In contrast, both the small and large noise regimes resist memorization, but through fundamentally different mechanisms: small noise avoids memorization due to limited training coverage, while large noise exhibits low posterior concentration and admits a provably near linear Gaussian denoising behavior. For the medium noise regime, we identify geometric conditions through which we propose a geometry-informed targeted intervention that mitigates memorization.

Motivation & Objective

• Motivate and analyze memorization vs. generalization in diffusion models using a geometric lens.
• Characterize how posterior weight concentration and Gaussian shell coverage vary across noise scales.
• Identify a 'danger zone' in medium noise where memorization is most likely to occur.
• Propose a mitigation strategy by selectively undertraining the intermediate noise regime.
• Validate the framework with empirical experiments on CIFAR-10 and related datasets.

Proposed method

• Define posterior weights m_sigma as the empirical optimal denoiser with weights w_i(x,sigma) (Eq. 2 and Eq. 3).
• Introduce Gaussian shell coverage to model supervised regions during training (S_sigma(x) and coverage C_sigma).
• Partition the noise schedule into three regimes based on posterior concentration and coverage: small, medium (danger zone), large.
• Analyze memorization via trajectory-level and per-noise-level metrics, including d_1NN/d_2NN tests for memorization.
• Provide theoretical results (Theorems 4.2, 4.8, 4.9, 4.11) describing weight concentration thresholds, coverage bounds, and large-sigma linear denoising.
• Suggest a practical mitigation by undertraining the medium noise regime and validate with denoiser swapping experiments.

Figure 1 : MSE to Clean Image. Comparison of denoising quality across noise levels. Solid lines: training data; dotted lines: test data. EDM-1K shows a generalization gap in the mid- $\sigma$ region.

Experimental results

Research questions

• RQ1 At which noise levels does memorization arise in diffusion models, and what geometric mechanisms drive it?
• RQ2 How do posterior weight concentration and Gaussian shell coverage vary across the noise schedule, and how do they interact to create a danger zone?
• RQ3 Can memorization be mitigated by targeting training in the intermediate noise regime without sacrificing generation quality?
• RQ4 How do trajectory-level memorization and per-noise-level memorization relate across different training data regimes?

Key findings

Condition	Noise Range (σ)	Mem. Rate
EDM-1K (default sample)	[0.002,80]	92.2%
EDM-50K (default sample)	[0.002,80]	0.0%
EDM-1K → EDM-50K swap: large region	σ>8.4	93.0%
EDM-1K → EDM-50K swap: medium region	[0.14,8.4]	0.0%
EDM-1K → EDM-50K swap: small region	σ<0.14	91.0%
EDM-50K → EDM-1K swap: large region	σ>8.4	0.0%
EDM-50K → EDM-1K swap: medium region	[0.14,8.4]	92.2%
EDM-50K → EDM-1K swap: small region	σ<0.14	0.0%

• Memorization risk is non-uniform across noise levels, peaking in a medium noise regime termed the danger zone.
• Small-noise regime resists memorization due to limited coverage, while large-noise regime resists memorization due to weak posterior concentration and near-linear denoising.
• Per-noise-level memorization concentrates in the intermediate regime, and denoiser swapping in this region flips memorization behavior.
• Empirical results show a sharp transition in posterior weight concentration and Gaussian shell coverage around moderate noise levels, aligning with memorization risk.
• Denoiser swapping experiments demonstrate that targeting the intermediate regime can mitigate memorization while preserving generation quality.
• Theoretical results (Theorems 4.2, 4.8, 4.9, 4.11) formalize the weight concentration, coverage behavior, and limiting linear denoising in large-noise regimes.

Figure 2 : Per-Noise-Level Memorization Rate. Fraction of denoised test images classified as memorized at each noise level.