[论文解读] Exposing the Fake: Effective Diffusion-Generated Images Detection

Ruipeng Ma, Jinhao Duan+3 more|arXiv (Cornell University)|Jul 12, 2023

Generative Adversarial Networks and Image Synthesis被引用 10

一句话总结

SeDID 利用扩散模型中的逐步反向去噪误差来检测扩散生成的图像，拥有两个分支（Statistical 与 NN-based），在 CIFAR10、TinyImageNet 和 CelebA 上超越了先前的方法。

ABSTRACT

Image synthesis has seen significant advancements with the advent of diffusion-based generative models like Denoising Diffusion Probabilistic Models (DDPM) and text-to-image diffusion models. Despite their efficacy, there is a dearth of research dedicated to detecting diffusion-generated images, which could pose potential security and privacy risks. This paper addresses this gap by proposing a novel detection method called Stepwise Error for Diffusion-generated Image Detection (SeDID). Comprising statistical-based $ ext{SeDID}_{ ext{Stat}}$ and neural network-based $ ext{SeDID}_{ ext{NNs}}$, SeDID exploits the unique attributes of diffusion models, namely deterministic reverse and deterministic denoising computation errors. Our evaluations demonstrate SeDID's superior performance over existing methods when applied to diffusion models. Thus, our work makes a pivotal contribution to distinguishing diffusion model-generated images, marking a significant step in the domain of artificial intelligence security.

研究动机与目标

• Motivate the need to detect diffusion-generated images and address limitations of prior methods that rely only on initial-step reconstruction.
• Propose SeDID to exploiting deterministic reverse and denoising errors across intermediate diffusion steps.
• Adapt a membership-inference attack perspective to emphasize distributional differences between real and generated data.
• Provide dual detection branches (statistical and neural-network) and evaluate on multiple datasets.
• Show that SeDID achieves superior detection performance over existing approaches.

提出的方法

• Formulate deterministic denoising and reverse functions (ψθ and φθ) and the t,δ-error measuring the discrepancy between reverse and denoise samples (E_{t,δ}).
• Define Stepwise Error Calculation Time Step T_SE and stepsize δ to compute E_{T_SE,δ} at chosen diffusion timesteps.
• Develop SeDID in two variants: SeDID_Stat (threshold-based classification using E_{T_SE,δ}) and SeDID_NNs (ResNet-18 trained to predict real vs synthetic from intermediate diffusion outputs).
• Use AUC, ACC, and TPR@FPR as evaluation metrics for both branches.
• Ground the method in equations for DDPM/DDIM-like processes and adapt to Latent Diffusion Model (LDM) in Appendix.
• Incorporate ideas from SecMI-inspired membership inference to emphasize distributional disparities between real and generated data.

实验结果

研究问题

• RQ1 Can real vs diffusion-generated images be distinguished by exploiting distributional disparities between natural and diffusion-synthesized visuals?
• RQ2 Do intermediate steps in the diffusion process contain informative signals beyond the final inversion at x0 for detection?
• RQ3 Does SeDID outperform existing diffusion-generated image detectors such as DIRE and SecMI-based approaches?

主要发现

T	CIFAR10 AUC	CIFAR10 ACC	TinyImageNet AUC	TinyImageNet ACC	CelebA AUC	CelebA ACC
165	0.4954	0.5089	0.1278	0.5000	0.9985	0.9843
330	0.5415	0.5279	0.0606	0.5000	0.4213	0.5103
495	0.5695	0.5579	0.5125	0.5240	0.2240	0.5001
660	0.5667	0.5845	0.4971	0.6059	0.1866	0.5489
825	0.8650	0.7992	0.9827	0.9615	0.0001	0.5000
990	0.8875	0.8244	0.9998	0.9966	0.0000	0.5000

• SeDID variants achieve higher AUC and ACC than the existing method across all three datasets (CIFAR10, TinyImageNet, CelebA).
• SeDID_Stat: CIFAR10 AUC 0.8874, ACC 0.8244; TinyImageNet AUC 0.9266, ACC 0.9004; CelebA AUC 0.9983, ACC 0.9825.
• SeDID_NNs: CIFAR10 AUC 0.8903, ACC 0.8218; TinyImageNet AUC 0.9999, ACC 0.9980; CelebA AUC 1.0000, ACC 1.0000.
• Average comparison (across datasets) shows substantial gains over the baseline method (Matsumoto et al. 2023).
• Optimal detection performance varies with T_SE and δ, with δ = 165 yielding best results across datasets.
• On Table 1 results, detection performance improves with larger diffusion time T for CIFAR10 and TinyImageNet, while CelebA shows different trends at higher T.