[Paper Review] Measuring Neural Net Robustness with Constraints

Osbert Bastani, Yani Ioannou+4 more|arXiv (Cornell University)|Jan 1, 2016

Adversarial Robustness in Machine Learning17 references115 citations

TL;DR

This paper proposes a constraint-based linear programming approach to measure neural network robustness by estimating the minimal adversarial perturbation (in L∞ norm) around a given input. It introduces two robustness metrics—adversarial frequency and severity—and demonstrates that models fine-tuned using their method are more robust across multiple evaluation criteria, unlike models overfitting to adversarial examples from prior algorithms.

ABSTRACT

Despite having high accuracy, neural nets have been shown to be susceptible to adversarial examples, where a small perturbation to an input can cause it to become mislabeled. We propose metrics for measuring the robustness of a neural net and devise a novel algorithm for approximating these metrics based on an encoding of robustness as a linear program. We show how our metrics can be used to evaluate the robustness of deep neural nets with experiments on the MNIST and CIFAR-10 datasets. Our algorithm generates more informative estimates of robustness metrics compared to estimates based on existing algorithms. Furthermore, we show how existing approaches to improving robustness "overfit" to adversarial examples generated using a specific algorithm. Finally, we show that our techniques can be used to additionally improve neural net robustness both according to the metrics that we propose, but also according to previously proposed metrics.

Motivation & Objective

• To address the lack of objective, reliable metrics for measuring neural network robustness against adversarial examples.
• To formalize pointwise robustness as a measurable property using L∞ distance to the nearest adversarial example.
• To develop a scalable and accurate algorithm for estimating robustness that avoids overfitting to adversarial examples generated by specific search algorithms.
• To evaluate whether models fine-tuned for robustness using existing methods actually generalize beyond the specific adversarial examples used in training.
• To demonstrate that robustness improvements can be achieved not only according to the proposed metrics but also in alignment with prior metrics.

Proposed method

• Formalize robustness ρ(f, x*) as the minimal L∞ norm of a perturbation r such that f(x* + r) ≠ f(x*), encoding this as a constrained optimization problem.
• Approximate the intractable robustness computation by restricting the search space to a convex, piecewise-linear region Z(x*) around x* where the network is linear.
• Model the robustness estimation as a linear program (LP), leveraging the piecewise linear structure of ReLU-based networks.
• Introduce an iterative constraint-solving optimization that accelerates LP solving by an order of magnitude compared to baseline methods.
• Use the resulting algorithm (ALP) to estimate robustness metrics—adversarial frequency and severity—across multiple datasets and models.
• Fine-tune models using adversarial examples generated by ALP and compare performance against models fine-tuned using L-BFGS-B and other baselines.

Experimental results

Research questions

• RQ1 Can robustness be measured objectively using a formal, quantifiable metric that captures both frequency and severity of adversarial examples?
• RQ2 Does the robustness of a neural network trained using adversarial examples depend on the specific algorithm used to generate them?
• RQ3 Can a constraint-based linear programming approach provide more accurate robustness estimates than existing optimization-based methods like L-BFGS-B?
• RQ4 To what extent do models fine-tuned for robustness overfit to adversarial examples generated by a particular algorithm?
• RQ5 Can the proposed method be scaled to deeper networks like NiN on CIFAR-10, and does it reveal previously undetected vulnerabilities?

Key findings

• The proposed ALP algorithm produces significantly more accurate robustness estimates than the L-BFGS-B baseline, especially in detecting the true frequency and severity of adversarial examples.
• Models fine-tuned using the L-BFGS-B algorithm exhibit overfitting: they are less robust under the ALP metric, despite appearing more robust under the baseline algorithm.
• The ALP-fine-tuned model shows substantially lower adversarial frequency (59.6%) and improved adversarial severity (3.88 pixels) compared to the original NiN (61.5% frequency, 2.82 pixels severity), indicating genuine robustness gains.
• On LeNet, the ALP-fine-tuned model outperforms the L-BFGS-B-fine-tuned model in both adversarial frequency and severity metrics, confirming that the former generalizes better across adversarial search methods.
• The ALP algorithm achieves a 10x speedup over a naive LP solver through iterative constraint handling, making robustness evaluation feasible for deeper networks.
• The NiN model on CIFAR-10 remains highly vulnerable to adversarial examples, with a 61.5% adversarial frequency, suggesting that current methods are insufficient for highly accurate robustness improvement at scale.