[论文解读] Understanding Hallucinations in Diffusion Models through Mode Interpolation

Sumukh K Aithal, Pratyush Maini+2 more|arXiv (Cornell University)|Jun 13, 2024

Mental Health Research Topics被引用 5

一句话总结

论文识别了扩散模型中的一个失败模式，称为模式插值，在该模式下模型在相邻数据模态之间生成样本，产生超出训练支撑的幻觉，并提出一个基于方差的度量在生成与递归训练期间检测并修剪此类样本。

ABSTRACT

Colloquially speaking, image generation models based upon diffusion processes are frequently said to exhibit "hallucinations," samples that could never occur in the training data. But where do such hallucinations come from? In this paper, we study a particular failure mode in diffusion models, which we term mode interpolation. Specifically, we find that diffusion models smoothly "interpolate" between nearby data modes in the training set, to generate samples that are completely outside the support of the original training distribution; this phenomenon leads diffusion models to generate artifacts that never existed in real data (i.e., hallucinations). We systematically study the reasons for, and the manifestation of this phenomenon. Through experiments on 1D and 2D Gaussians, we show how a discontinuous loss landscape in the diffusion model's decoder leads to a region where any smooth approximation will cause such hallucinations. Through experiments on artificial datasets with various shapes, we show how hallucination leads to the generation of combinations of shapes that never existed. Finally, we show that diffusion models in fact know when they go out of support and hallucinate. This is captured by the high variance in the trajectory of the generated sample towards the final few backward sampling process. Using a simple metric to capture this variance, we can remove over 95% of hallucinations at generation time while retaining 96% of in-support samples. We conclude our exploration by showing the implications of such hallucination (and its removal) on the collapse (and stabilization) of recursive training on synthetic data with experiments on MNIST and 2D Gaussians dataset. We release our code at https://github.com/locuslab/diffusion-model-hallucination.

研究动机与目标

• 形式化并表征扩散模型中的幻觉为相邻数据模态之间的模式插值。
• 分析学习到的分数函数如何平滑不连续性，导致在不相交模态之间的插值样本与支撑外样本。
• 提出一个基于轨迹方差的度量，在生成时检测并过滤幻觉。
• 探讨对递归训练的影响，并通过对合成数据和 MNIST 数据进行预先过滤来证明缓解方法的有效性。

提出的方法

• 研究一维和二维高斯混合以展示扩散模型在相邻模态之间插值。
• 表明神经网络学习的是对真实分数函数的平滑近似，导致在不相连的模态之间发生插值。
• 在最终反向扩散步骤中识别预测 x0 的轨迹方差较高的轨迹，作为幻觉的标志。
• 定义基于不同时间步的预测 x0 的方差的幻觉度量 Hal(x) 来对样本进行分类。
• 评估该度量的过滤能力：在保持约 95–98% 的支撑内样本的同时，移除约 95–96% 的幻觉样本。

Figure 1 : Hallucinations in Diffusion Models : Original Dataset (Left) & Generated Dataset (Right). The original dataset consists of 64x64 images divided into three columns, each containing a triangle, square, or pentagon with a 0.5 probability of the shape being present. Each shape appears at most

实验结果

研究问题

• RQ1 是什么原因导致扩散模型生成位于训练支撑之外的样本（幻觉）？
• RQ2 扩散模型是否在接近的数据模态之间表现出模式插值，分数函数如何贡献？
• RQ3 基于轨迹方差的度量是否能在不严重损害支撑内样本的前提下检测并过滤幻觉？
• RQ4 幻觉对递归训练和模型稳定性有什么影响？

主要发现

• 扩散模型在合成的一维和二维高斯混合中在相邻模态之间插值，产生超出训练支撑的样本。
• 平滑学习到的分数函数，而非尖锐的模态跃迁，驱动不相连模态之间的插值。
• 预测的 x0 轨迹在反向扩散末期的高方差与幻觉相关，且能实现检测。
• Hal(x) 度量在多种设置下可移除约 95–96% 的幻觉，同时保留约 95–98% 的支撑内样本。
• 基于该度量的预先过滤在对二维高斯、简单形状和 MNIST 数据集进行递归训练时可缓解模型崩溃问题。

Figure 2 : Mode Interpolation in 1D Gaussian . The red curve indicates the PDF of the true data distribution $q(x)$ , which is a mixture of 3 Gaussians (notice that the y-axis is in log-scale). In blue, we show a density histogram of the samples generated by a DDPM trained on varying number of sampl