[Paper Review] advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch

Gavin Weiguang Ding, Luyu Wang+1 more|arXiv (Cornell University)|Feb 20, 2019

Adversarial Robustness in Machine Learning22 references143 citations

TL;DR

advertorch v0.1 is a PyTorch-based toolbox providing implementations of adversarial attacks, defenses, and robust training methods, with a BPDA wrapper and clear design for attack-in-the-loop research.

ABSTRACT

advertorch is a toolbox for adversarial robustness research. It contains various implementations for attacks, defenses and robust training methods. advertorch is built on PyTorch (Paszke et al., 2017), and leverages the advantages of the dynamic computational graph to provide concise and efficient reference implementations. The code is licensed under the LGPL license and is open sourced at https://github.com/BorealisAI/advertorch .

Motivation & Objective

• Provide a concise, consistent API for attacks and defenses to streamline adversarial robustness research.
• Offer concise, reference implementations leveraging PyTorch dynamic graphs for fast execution.
• Support attacks-in-the-loop workflows (e.g., adversarial training) with GPU acceleration.
• Incorporate preprocessing defenses and robust training approaches with modular design.
• Establish versioning and reporting practices to enable reproducible benchmark results.

Proposed method

• Implement gradient-based attacks with modular core components: predict function, loss_fn, and perturb method.
• Include additional attacks beyond gradient-based methods (e.g., SinglePixelAttack, LocalSearchAttack, JacobianSaliencyMapAttack).
• Provide BPDAWrapper to substitute backward passes for non-differentiable or gradient-obfuscating defenses.
• Incorporate preprocessing-based defenses as PyTorch modules (e.g., JPEGFilter, BitSqueezing, smoothing filters).
• Offer robust training references, including Madry-style adversarial training and provably robust training as examples in advertorch_examples.
• Follow Semantic Versioning 2.0.0 and document benchmark hyperparameters for reproducibility.

Experimental results

Research questions

• RQ1 What attacks are implemented in advertorch v0.1 and how are they structured for flexibility?
• RQ2 How does BPDAWrapper enable attacks against defended models with non-differentiable components?
• RQ3 What preprocessing-based defenses are provided and how can they be composed?
• RQ4 What robust training approaches are demonstrated, and how are they organized for reference implementations?
• RQ5 How should benchmark results be reported to ensure reproducibility across versions?

Key findings

• The toolbox provides a range of gradient-based attacks and other attack types with a decoupled core design (predict, loss_fn, perturb).
• BPDAWrapper enables gradient-based attacks against defended models by substituting backward passes, including straight-through estimators.
• Preprocessing-based defenses are implemented as modular PyTorch components that can be composed on the fly.
• Robust training references are included, with an example implementation of Madry-style adversarial training in advertorch_examples.
• The project adheres to Semantic Versioning and emphasizes detailed hyperparameter reporting for benchmarks.