[Paper Review] DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing

Jiawei Zhang, Zhong‐Zhu Chen+3 more|arXiv (Cornell University)|Aug 28, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

This paper proposes DiffSmooth, a certifiably robust learning framework that combines diffusion model-based adversarial purification with local smoothing to achieve state-of-the-art certified robustness. By first denoising perturbed inputs via a one-shot DDPM and then applying local smoothing, DiffSmooth improves certified accuracy—achieving 53.0% under ℓ₂ radius 1.5 on ImageNet, a significant improvement over prior SOTA at 36.0%.

ABSTRACT

Diffusion models have been leveraged to perform adversarial purification and thus provide both empirical and certified robustness for a standard model. On the other hand, different robustly trained smoothed models have been studied to improve the certified robustness. Thus, it raises a natural question: Can diffusion model be used to achieve improved certified robustness on those robustly trained smoothed models? In this work, we first theoretically show that recovered instances by diffusion models are in the bounded neighborhood of the original instance with high probability; and the "one-shot" denoising diffusion probabilistic models (DDPM) can approximate the mean of the generated distribution of a continuous-time diffusion model, which approximates the original instance under mild conditions. Inspired by our analysis, we propose a certifiably robust pipeline DiffSmooth, which first performs adversarial purification via diffusion models and then maps the purified instances to a common region via a simple yet effective local smoothing strategy. We conduct extensive experiments on different datasets and show that DiffSmooth achieves SOTA-certified robustness compared with eight baselines. For instance, DiffSmooth improves the SOTA-certified accuracy from $36.0\%$ to $53.0\%$ under $\ell_2$ radius $1.5$ on ImageNet. The code is available at [https://github.com/javyduck/DiffSmooth].

Motivation & Objective

• Address the limited certified robustness of existing diffusion-based defenses under large perturbation radii.
• Overcome the high computational cost of multi-shot reverse diffusion in prior methods like DensePure.
• Improve certified robustness of smoothed classifiers by leveraging the denoising properties of diffusion models and local smoothing.
• Provide theoretical justification for the boundedness of recovered instances and the approximation of original inputs via one-shot DDPM.
• Develop a practical, efficient pipeline that maintains high benign accuracy while achieving superior certified robustness.

Proposed method

• Apply a one-shot denoising step using a pre-trained DDPM to purify adversarial or Gaussian-augmented inputs, recovering a denoised instance close to the original.
• Theoretical analysis shows that the denoised output lies within a bounded neighborhood of the original input with high probability under mild conditions.
• Use the recovered instance as input to a local smoothing strategy, where multiple noisy variants are generated and classified via a robustly trained smoothed classifier.
• Apply majority voting over the predictions of the smoothed classifier on the locally perturbed denoised instances to provide probabilistic certification.
• Optimize the magnitude of local smoothing noise (σ′) and number of smoothing samples (m) to maximize certified robustness without degrading performance.
• Maintain a fixed number of prediction queries (e.g., 10,000 on ImageNet) to ensure fair comparison with baselines like DDS.

Experimental results

Research questions

• RQ1 Can diffusion-based purification improve certified robustness for smoothed classifiers under large perturbation radii?
• RQ2 Does combining diffusion-based denoising with local smoothing yield better certified robustness than relying solely on diffusion or randomized smoothing?
• RQ3 How does the choice of local smoothing noise magnitude (σ′) affect certified accuracy and benign accuracy?
• RQ4 Can a one-shot DDPM approximation effectively recover the original input distribution, enabling robustness certification?
• RQ5 Is the computational bottleneck of the method dominated by the diffusion step or the local smoothing step?

Key findings

• DiffSmooth achieves a certified accuracy of 53.0% under ℓ₂ radius 1.5 on ImageNet, significantly improving upon the prior SOTA of 36.0%.
• With the same computation cost as DDS (10,000 prediction queries on ImageNet), DiffSmooth achieves higher certified robustness and benign accuracy.
• Using only m=5 local smoothing samples achieves non-trivial certified robustness, indicating that the method is computationally efficient beyond the diffusion step.
• The method’s performance is sensitive to the choice of σ′, with optimal values depending on the underlying smoothed model (e.g., σ′ ≈ σ for Gaussian smoothing, σ′ ≈ σ/2 for SmoothAdv).
• The main computational cost stems from the reverse diffusion step (553s per image on ImageNet), not from local smoothing, which is relatively lightweight.
• Local smoothing alone cannot improve certified accuracy over standard randomized smoothing, and may degrade performance if σ′ is too large, validating the necessity of the diffusion purification step.