[论文解读] Robust Pre-Processing: A Robust Defense Method Against Adversary Attack

Adnan Siraj Rakin, Zhezhi He+2 more|arXiv (Cornell University)|Feb 5, 2018

Adversarial Robustness in Machine Learning被引用 4

一句话总结

本文提出了一种鲁棒预处理方法，这是一种防御机制，通过使用双曲正切（Tanh）激活、平滑处理和批量归一化来增强模型对对抗攻击的鲁棒性。该方法在白盒攻击下于MNIST数据集上实现了98.7%的准确率，超越了当前最先进的防御方法，且无需对抗训练，同时保持了较高的干净数据准确率。

ABSTRACT

Deep learning algorithms and networks are vulnerable to perturbed inputs which are known as the adversarial attack. Many defense methodologies have been investigated to defend such adversarial attack. In this work, we propose a novel methodology to defend the existing powerful attack model. Such attack models have achieved record success against MNIST dataset to force it to miss-classify all of its inputs. Whereas Our proposed defense method robust pre-processing achieves the best accuracy among the current state of the art defenses. It consists of Tanh (hyperbolic tangent) function, smoothing and batch normalization to process the input data which will make it more robust over the adversarial attack. robust pre-processing improves the white box attack accuracy of MNIST from 94.3% to 98.7%. Even with increasing defense when others defenses completely fail, robust pre-processing remains one of the strongest ever reported. Another strength of our defense is that it eliminates the need for adversarial training as it can significantly increase the MNIST accuracy without adversarial training as well. This makes it a more generalized defense method with almost half training overhead and much-improved accuracy. robust pre-processing can also increase the inference accuracy in the face of the powerful attack on CIFAR-10 and SVHN data set as well without much sacrificing clean data accuracy.

研究动机与目标

• 解决深度学习模型对微小、难以察觉的输入扰动所引发的对抗攻击的脆弱性问题。
• 开发一种无需依赖对抗训练的防御机制，以降低训练开销。
• 在显著提升对强大白盒攻击的鲁棒性的同时，保持对干净数据的高准确率。
• 在多个数据集（包括MNIST、CIFAR-10和SVHN）上实现良好泛化，且对干净输入的性能下降最小。

提出的方法

• 在将数据输入模型之前，使用双曲正切（Tanh）函数对输入数据进行归一化和稳定处理。
• 引入平滑操作，以降低对对抗样本典型的小幅输入扰动的敏感性。
• 使用批量归一化进一步稳定输入分布，并在推理过程中提升泛化能力。
• 通过结合Tanh、平滑处理和批量归一化的预处理流水线处理原始输入，再进行模型推理。
• 作为独立的插件模块运行，与主模型架构解耦，具备广泛的兼容性。
• 通过输入变换实现内在鲁棒性，避免了对抗训练。

实验结果

研究问题

• RQ1 是否可以通过预处理流水线在不使用对抗训练的情况下提升对强白盒对抗攻击的鲁棒性？
• RQ2 鲁棒预处理在多个基准数据集上对干净数据准确率的影响如何？
• RQ3 在白盒攻击下，鲁棒预处理在MNIST上相较于现有最先进防御方法的性能提升程度如何？
• RQ4 该方法是否能在CIFAR-10和SVHN等其他数据集上保持干净数据准确率的同时实现良好泛化？

主要发现

• 鲁棒预处理将MNIST在白盒攻击下的准确率从94.3%提升至98.7%，优于所有现有最先进防御方法。
• 该方法在MNIST、CIFAR-10和SVHN上均保持了高干净数据准确率，且无需对抗训练。
• 与基于对抗训练的防御方法相比，训练开销减少了约一半。
• 即使在攻击强度不断增强时，其他防御方法失效的情况下，该防御仍保持有效性。
• 预处理流水线在多个数据集上均提升了鲁棒性，证明了其在MNIST之外的泛化能力。
• Tanh、平滑处理与批量归一化的结合形成了一种稳定且鲁棒的输入变换，能有效抵抗对抗扰动。