[Paper Review] Foolbox: A Python toolbox to benchmark the robustness of machine learning models

Jonas Rauber, Wieland Brendel+1 more|arXiv (Cornell University)|Jul 13, 2017

Computational Physics and Python ApplicationsComputer Science143 citations

TL;DR

Foolbox provides reference implementations of numerous adversarial attacks and a framework to benchmark model robustness by finding the minimum perturbation required to misclassify, across multiple frameworks and criteria.

ABSTRACT

Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at https://github.com/bethgelab/foolbox . The most up-to-date documentation can be found at http://foolbox.readthedocs.io .

Motivation & Objective

• Motivate the need to quantify robustness of ML models against adversarial perturbations using minimum perturbations as a benchmarking metric.
• Provide a unified, framework-agnostic interface to generate adversarial examples and compare robustness across models.
• Offer reference implementations of many attacks with internal hyperparameter tuning to approximate minimum perturbations.
• Enable reporting of benchmark results with standardized versioning and criteria to ensure comparability.

Proposed method

• Define a five-part structure for adversarial crafting: model, criterion, distance, attack, and adversarial object.
• Interface with multiple ML frameworks through dedicated model adapters (TensorFlow, PyTorch, Theano, Lasagne, Keras, MXNet).
• Provide a CompositeModel to attack non-differentiable models by combining predictions and gradients from different sources.
• Implement a suite of adversarial criteria (misclassification, top-k, target-based criteria) and multiple distance measures (L1, L2/mean squared, L-infinity, L0).
• Include a large set of attacks with internal hyperparameter tuning to locate minimum perturbations, using internal line-search or parameter adaptation where needed.

Experimental results

Research questions

• RQ1 How can a unified Python toolkit enable fair, cross-framework benchmarking of model robustness to adversarial perturbations?
• RQ2 What is the minimum perturbation concept for robustness benchmarking and how can it be reliably estimated across attacks?
• RQ3 Can different adversarial criteria and distance metrics be integrated to compare robustness comprehensively?
• RQ4 How should benchmark results be reported and versioned to ensure comparability over time?

Key findings

• Foolbox provides interfaces to major ML frameworks and supports many attacks with a consistent API.
• Attacks perform internal hyperparameter tuning to approximate the minimum adversarial perturbation.
• A CompositeModel enables attacks on non-differentiable models by combining model predictions with gradients from another source.
• The toolkit supports multiple adversarial criteria and distance measures to diversify robustness assessment.
• A structured reporting and semantic versioning scheme is recommended to ensure comparability of benchmark results.