[Paper Review] DIRE for Diffusion-Generated Image Detection

Zhendong Wang, Jianmin Bao+5 more|arXiv (Cornell University)|Mar 16, 2023

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper proposes DIRE (Diffusion Reconstruction Error), a novel image representation that measures reconstruction error between an input image and its reconstruction via a pre-trained diffusion model. DIRE enables highly accurate detection of diffusion-generated images by exploiting the fact that generated images reconstruct more accurately than real images, achieving state-of-the-art performance on unseen models and various perturbations.

ABSTRACT

Diffusion models have shown remarkable success in visual synthesis, but have also raised concerns about potential abuse for malicious purposes. In this paper, we seek to build a detector for telling apart real images from diffusion-generated images. We find that existing detectors struggle to detect images generated by diffusion models, even if we include generated images from a specific diffusion model in their training data. To address this issue, we propose a novel image representation called DIffusion Reconstruction Error (DIRE), which measures the error between an input image and its reconstruction counterpart by a pre-trained diffusion model. We observe that diffusion-generated images can be approximately reconstructed by a diffusion model while real images cannot. It provides a hint that DIRE can serve as a bridge to distinguish generated and real images. DIRE provides an effective way to detect images generated by most diffusion models, and it is general for detecting generated images from unseen diffusion models and robust to various perturbations. Furthermore, we establish a comprehensive diffusion-generated benchmark including images generated by eight diffusion models to evaluate the performance of diffusion-generated image detectors. Extensive experiments on our collected benchmark demonstrate that DIRE exhibits superiority over previous generated-image detectors. The code and dataset are available at https://github.com/ZhendongWang6/DIRE.

Motivation & Objective

• To address the critical need for reliable detection of diffusion-generated images, which pose privacy and deepfake risks.
• To overcome the limitations of existing detectors that fail to generalize across different diffusion models, even when trained on similar data.
• To develop a generalizable, robust, and efficient detection framework that works across unseen diffusion models and perturbations.
• To establish a comprehensive benchmark dataset for evaluating diffusion-generated image detectors.
• To demonstrate that reconstruction error differences between real and generated images can serve as a strong, generalizable detection signal.

Proposed method

• Propose DIRE as a new image representation computed as the absolute difference between the original image and its reconstruction by a pre-trained diffusion model.
• Use DDIM-based inversion and denoising to reconstruct input images, leveraging the fact that diffusion models can reconstruct their own generated outputs more accurately.
• Train a simple binary classifier on DIRE features to distinguish real from generated images, ensuring generalization to unseen models.
• Apply the absolute value operation on the reconstruction residual to significantly improve detection performance, especially on challenging models like Stable Diffusion v1.
• Use frequency and noise pattern analysis to qualitatively validate that DIRE captures low-level differences between real and generated images.
• Construct the DiffusionForensics benchmark dataset with images from eight diverse diffusion models (including unconditional, conditional, and text-to-image variants) on LSUN-Bedroom and ImageNet.

Experimental results

Research questions

• RQ1 Can a reconstruction error-based representation effectively distinguish between real and diffusion-generated images?
• RQ2 Does the DIRE representation generalize to diffusion models not seen during training?
• RQ3 How robust is the DIRE-based detector to common image perturbations such as noise, blur, and compression?
• RQ4 What is the impact of the number of DDIM sampling steps on DIRE-based detection performance?
• RQ5 Is DIRE a more effective input representation than raw RGB or reconstructed images for detection tasks?

Key findings

• DIRE achieves 99.7% accuracy and 100% average precision on unseen Stable Diffusion v1-generated images, significantly outperforming prior methods.
• The DIRE-based detector generalizes effectively to unseen diffusion models, including those not in the training data, due to the inherent reconstruction error difference between real and generated images.
• Applying the absolute value to the reconstruction residual improves detection accuracy by up to 12.7 percentage points on SD-v1, confirming its critical role.
• The method is robust to various perturbations such as Gaussian noise, blur, and JPEG compression, maintaining high detection performance.
• Qualitative analysis shows that DIRE captures low-level differences: real images exhibit more structured noise patterns and richer frequency spectra than generated images.
• Combining DIRE with raw RGB images degrades performance, indicating that DIRE alone is a more effective and sufficient representation for detection.