[論文レビュー] Fixing Data Augmentation to Improve Adversarial Robustness

Sylvestre-Alvise Rebuffi, Sven Gowal+4 more|arXiv (Cornell University)|Mar 2, 2021

Adversarial Robustness in Machine Learning参考文献 72被引用数 104

ひとこと要約

本論文は、ヒューリスティック駆動とデータ駆動の拡張をモデルウェイト平均と組み合わせることが、外部データを使用せずに CIFAR-10 における敵対的耐性を大幅に向上させることを示しており、l-infinity 8/255 に対する堅牢性正確度は 64.20%、l2 128/255 に対しては 80.38% である。

ABSTRACT

Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training. In this paper, we focus on both heuristics-driven and data-driven augmentations as a means to reduce robust overfitting. First, we demonstrate that, contrary to previous findings, when combined with model weight averaging, data augmentation can significantly boost robust accuracy. Second, we explore how state-of-the-art generative models can be leveraged to artificially increase the size of the training set and further improve adversarial robustness. Finally, we evaluate our approach on CIFAR-10 against $\ell_\infty$ and $\ell_2$ norm-bounded perturbations of size $ε= 8/255$ and $ε= 128/255$, respectively. We show large absolute improvements of +7.06% and +5.88% in robust accuracy compared to previous state-of-the-art methods. In particular, against $\ell_\infty$ norm-bounded perturbations of size $ε= 8/255$, our model reaches 64.20% robust accuracy without using any external data, beating most prior works that use external data.

研究の動機と目的

• Investigate robust overfitting in adversarial training and the role of model weight averaging (WA).
• Assess whether heuristics-driven augmentations (e.g., MixUp, CutMix, Cutout) can improve robustness when combined with WA.
• Evaluate data-driven augmentations using generative models to enlarge the effective training set without external data.
• Explore whether combining data-driven and heuristics-driven augmentations yields additive robustness gains.

提案手法

• Use TRADES with 10 PGD steps for adversarial training on CIFAR-10.
• Apply heuristics-driven augmentations (MixUp, Cutout, CutMix, RICAP) with WA and compare to padding/cropping baselines.
• Incorporate data-driven augmentations from generative models (DDPM, VDVAE, BigGAN) trained only on the original data.
• Tune original-to-generated data ratios per batch to maximize robustness on a validation set.
• Evaluate robustness against AutoAttack and MultiTargeted, reporting robust accuracy for l-infinity (epsilon=8/255) and l2 (epsilon=128/255).
• Analyze how DDPM, in particular, yields samples closest to the test distribution and how combining with CutMix improves results.

実験結果

リサーチクエスチョン

• RQ1 Can model weight averaging (WA) enhance robustness when robust overfitting is controlled or absent?
• RQ2 Do heuristics-driven augmentations (MixUp, CutMix, Cutout) improve robust accuracy under adversarial training when combined with WA?
• RQ3 Do data-driven augmentations from generative models improve adversarial robustness without external data, and which models are most effective?
• RQ4 Are heuristics-driven and data-driven augmentations complementary, and does their combination yield higher robust accuracy than either alone?

主な発見

• Data augmentation with WA yields robust accuracy improvements, contradicting some prior claims that augmentations beyond padding-and-cropping are ineffective.
• CutMix with WA achieves robust accuracy of 60.07% on CIFAR-10 against l-infinity 8/255 (higher than MixUp or Cutout under the same regime).
• DDPM-generated data (trained only on the original data) significantly improves robustness, achieving 63.58% robust accuracy against l-infinity 8/255 without external data.
• Combining CutMix with DDPM-generated data yields a new state-of-the-art 64.20% robust accuracy on CIFAR-10 for l-infinity 8/255 and 80.38% robust accuracy for l2 128/255 without external data.
• With external data (500k images from 80M-Ti), robustness improves further (e.g., 66.56% for l-infinity with large models).
• Generative data modeled by DDPM can outperform other sources of generated data in contributing to robustness, especially without external data.