[論文レビュー] Advbox: a toolbox to generate adversarial examples that fool neural networks

Dou Goodman, Xin Hao+4 more|arXiv (Cornell University)|Jan 13, 2020

Adversarial Robustness in Machine Learning参考文献 50被引用数 39

ひとこと要約

AdvBox は、複数のフレームワーク（PaddlePaddle、PyTorch、Caffe2、MXNet、Keras、TensorFlow）に跨ってニューラルネットワークを欺く敵対的サンプルを生成できるPythonベースのツールボックスであり、ブラックボックス攻撃と現実世界の攻撃シナリオをサポートするとともに、ロバストネスのベンチマークにも対応します。

ABSTRACT

In recent years, neural networks have been extensively deployed for computer vision tasks, particularly visual classification problems, where new algorithms reported to achieve or even surpass the human performance. Recent studies have shown that they are all vulnerable to the attack of adversarial examples. Small and often imperceptible perturbations to the input images are sufficient to fool the most powerful neural networks. \emph{Advbox} is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle, PyTorch, Caffe2, MxNet, Keras, TensorFlow and it can benchmark the robustness of machine learning models. Compared to previous work, our platform supports black box attacks on Machine-Learning-as-a-service, as well as more attack scenarios, such as Face Recognition Attack, Stealth T-shirt, and DeepFake Face Detect. The code is licensed under the Apache 2.0 and is openly available at https://github.com/advboxes/AdvBox. Advbox now supports Python 3.

研究の動機と目的

• Motivate and enable adversarial example generation and evaluation across major DL frameworks.
• Provide a unified platform for attacks, defenses, and robustness benchmarking.
• Extend attack scenarios to black-box ML-as-a-service and real-world settings (Face Recognition Attack, Stealth T-shirt, DeepFake Face Detect).

提案手法

• Implements multiple adversarial attacks (FGSM, BIM, DeepFool, JSMA, CW, PGD) with distance-based perturbation measurement.
• Provides model interfaces for TensorFlow, PyTorch, MXNet, PaddlePaddle and supports GraphPipe to shield underlying platforms.
• Encapsulates adversary, attack, and defense logic in an object-oriented AdvBox framework.
• Includes six defense methods (Feature Squeezing, Spatial Smoothing, Label Smoothing, Gaussian Augmentation, Adversarial Training, Thermometer Encoding).
• Offers a robustness evaluation sub-project Perceptron to benchmark model robustness across CV DNNs and cloud APIs.
• Surveys and enables attack scenarios beyond standard benchmarks (Face Recognition Attack, Stealth T-shirt, DeepFake Face Detect).

実験結果

リサーチクエスチョン

• RQ1 How can adversarial examples be generated efficiently across multiple deep learning frameworks?
• RQ2 What defenses are effective against a range of attacks, and how can robustness be benchmarked consistently?
• RQ3 Can adversarial attacks be extended to black-box, ML-as-a-service settings and real-world scenarios such as face recognition and DeepFake detection?
• RQ4 How do different perturbation metrics (L1, L2, L-infinity, etc.) influence attack effectiveness across tasks?

主な発見

• AdvBox supports adversarial generation across PaddlePaddle, PyTorch, Caffe2, MXNet, Keras, and TensorFlow with black-box options.
• It implements six core attacks (FGSM, BIM, DeepFool, JSMA, CW, PGD) and aligns them with common perturbation norms (L1, L2, L∞).
• The platform provides six defenses and demonstrates robustness evaluation via the Perceptron benchmark for vision models and cloud APIs.
• AdvBox enables attack scenarios beyond standard benchmarks, including Face Recognition Attack, Stealth T-shirt, and DeepFake Face Detect.
• AdvBox emphasizes accessibility and openness, with open-source availability and Python-based, object-oriented design.
• The toolbox supports GraphPipe for framework-agnostic model querying and black-box access to model files from diverse ecosystems.