[Paper Review] Diffusion Models for Adversarial Purification

Weili Nie, Brandon Guo+4 more|arXiv (Cornell University)|May 16, 2022

Adversarial Robustness in Machine Learning78 citations

TL;DR

DiffPure uses forward and reverse diffusion processes to purify adversarial images, enabling strong adaptive attacks to be evaluated via an adjoint gradient method, and achieves state-of-the-art robustness across CIFAR-10, ImageNet, and CelebA-HQ.

ABSTRACT

Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model. These methods do not make assumptions on the form of attack and the classification model, and thus can defend pre-existing classifiers against unseen threats. However, their performance currently falls behind adversarial training methods. In this work, we propose DiffPure that uses diffusion models for adversarial purification: Given an adversarial example, we first diffuse it with a small amount of noise following a forward diffusion process, and then recover the clean image through a reverse generative process. To evaluate our method against strong adaptive attacks in an efficient and scalable way, we propose to use the adjoint method to compute full gradients of the reverse generative process. Extensive experiments on three image datasets including CIFAR-10, ImageNet and CelebA-HQ with three classifier architectures including ResNet, WideResNet and ViT demonstrate that our method achieves the state-of-the-art results, outperforming current adversarial training and adversarial purification methods, often by a large margin. Project page: https://diffpure.github.io.

Motivation & Objective

• Motivate robust defense against unseen adversaries without re-training the classifier.
• Introduce diffusion-based adversarial purification to wash out perturbations while preserving semantics.
• Provide theoretical guidance on how diffusion timestep balances purification and semantics.
• Enable efficient gradient computation through adjoint methods for strong adaptive attacks.
• Demonstrate empirically that DiffPure achieves superior robustness across multiple datasets and architectures.

Proposed method

• Apply a forward diffusion step to add a small amount of noise to an adversarial image.
• Diffuse to a timestep t* to remove local perturbations while preserving label semantics.
• Recover a purified image by solving the reverse SDE from t* to 0 using the denoising model.
• Use the adjoint method to backpropagate gradients through the SDE for evaluating against adaptive attacks.
• Provide theoretical bounds on the impact of diffusion timesteps on purification and semantics.
• Evaluate with AutoAttack and BPDA+EOT across CIFAR-10, ImageNet, and CelebA-HQ on ResNet, WideResNet, and ViT.

Experimental results

Research questions

• RQ1 How can diffusion models purify adversarial perturbations without assuming attack form or classifier?
• RQ2 What is the optimal diffusion timestep t* that balances removing perturbations and preserving data semantics?
• RQ3 Can adjoint-based gradient computation enable robust evaluation against strong adaptive attacks?
• RQ4 Do diffusion-based purifications achieve state-of-the-art robustness across diverse datasets and architectures?
• RQ5 How does DiffPure compare to adversarial training and other purifying methods under unseen threat models?

Key findings

• DiffPure achieves state-of-the-art robustness, improving adversarial robustness over AutoAttack ℓ∞ on CIFAR-10 by up to 5.44 percentage points and on ImageNet by up to 7.68 points.
• On CIFAR-10 ℓ2, DiffPure reaches 78.58% robust accuracy (versus ~) and surpasses methods without extra data in several settings.
• DiffPure attains large gains against unseen threats, with improvements up to 36% robust accuracy over competitive unseen-threat baselines.
• Against BPDA+EOT, DiffPure yields substantial robust-accuracy gains (e.g., +11.31% on CIFAR-10 and +15.63% on CelebA-HQ).
• For CelebA-HQ and CIFAR-10, diffusion-based purifications outperform ND-based generative purifiers in both standard and robust metrics under strong adaptive attacks.
• Ablation shows optimal t* is small and varies by threat model, with standard accuracy decreasing as t* grows but robust accuracy peaking at an intermediate t*.