[Paper Review] Robustness of classifiers: from adversarial to random noise

Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli+1 more|arXiv (Cornell University)|Aug 31, 2016

Adversarial Robustness in Machine Learning17 references119 citations

TL;DR

The paper analyzes classifier robustness under a semi-random noise regime that interpolates random and adversarial perturbations, showing robustness bounds depend on boundary curvature and data/subspace dimensions, and confirming empirical validity on deep nets.

ABSTRACT

Several recent works have shown that state-of-the-art classifiers are vulnerable to worst-case (i.e., adversarial) perturbations of the datapoints. On the other hand, it has been empirically observed that these same classifiers are relatively robust to random noise. In this paper, we propose to study a extit{semi-random} noise regime that generalizes both the random and worst-case noise regimes. We propose the first quantitative analysis of the robustness of nonlinear classifiers in this general noise regime. We establish precise theoretical bounds on the robustness of classifiers in this general regime, which depend on the curvature of the classifier's decision boundary. Our bounds confirm and quantify the empirical observations that classifiers satisfying curvature constraints are robust to random noise. Moreover, we quantify the robustness of classifiers in terms of the subspace dimension in the semi-random noise regime, and show that our bounds remarkably interpolate between the worst-case and random noise regimes. We perform experiments and show that the derived bounds provide very accurate estimates when applied to various state-of-the-art deep neural networks and datasets. This result suggests bounds on the curvature of the classifiers' decision boundaries that we support experimentally, and more generally offers important insights onto the geometry of high dimensional classification problems.

Motivation & Objective

• Motivate and formalize robustness under noise regimes that interpolate random and adversarial perturbations.
• Define a curvature-based framework to analyze binary and multi-class decision boundaries.
• Derive bounds linking semi-random robustness to adversarial robustness in terms of data dimension and subspace dimension.
• Provide theoretical guarantees for nonlinear classifiers and affine classifiers within the semi-random noise model.
• Empirically validate theoretical bounds on state-of-the-art deep networks and visualize boundary curvature.

Proposed method

• Define robustness r_S^* as the minimal perturbation within a subspace S that changes the predicted label.
• Introduce random and semi-random noise regimes via subspace dimension m and random subspace selection.
• Derive bounds for affine classifiers showing r_S^* scales as sqrt(d/m) times the adversarial perturbation r^*.
• Extend to nonlinear classifiers using a curvature measure of pairwise decision boundaries and prove analogous bounds under small curvature.
• Define and analyze a curvature κ(B_{i,j}) based on inscribed balls along decision boundaries.
• Provide corollaries and practical curvature conditions that ensure the random/semi-random robustness bound holds for multi-class settings.

Experimental results

Research questions

• RQ1 How does robustness under random and semi-random noise relate to adversarial robustness for affine and nonlinear classifiers?
• RQ2 What role does the curvature of pairwise decision boundaries play in bounding semi-random robustness?
• RQ3 Can we interpolate between random-noise robustness and worst-case robustness via the subspace dimension m?
• RQ4 Do empirical deep networks exhibit the predicted curvature-related robustness properties across datasets?
• RQ5 How can one visualize and estimate boundary curvature to support the theoretical bounds?

Key findings

• Robustness to random noise scales roughly as sqrt(d) times the adversarial distance for affine classifiers when the boundary curvature is small.
• In the semi-random regime, robustness scales as sqrt(d/m) times the adversarial distance, interpolating between random and worst-case regimes.
• For nonlinear classifiers, robustness bounds hold under a bounded curvature κ(B_{i,j}) of pairwise boundaries.
• Corollaries show that with curvature constraints on near-class boundaries, r_S^* is within a constant factor of sqrt(d/m) times r^*, similar to linear cases.
• Experiments on state-of-the-art networks (e.g., VGG-F, VGG-19, LeNet variants) show the predicted beta(f;m) values close to 1 across various m/d settings, supporting the theory.
• Visualizations suggest small curvature except at junctions, aligning with the curvature-based assumptions.