[Paper Review] Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

Yizhen Wang, Somesh Jha+1 more|arXiv (Cornell University)|Jun 13, 2017

Adversarial Robustness in Machine Learning26 references63 citations

TL;DR

The paper develops a bias-variance–style framework for robustness to adversarial examples and analyzes k-nearest neighbors, revealing regime-dependent robustness and proposing a robust 1-NN variant with theoretical guarantees and empirical validation.

ABSTRACT

Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training samples remains ill-understood. In this work, we introduce a theoretical framework analogous to bias-variance theory for understanding these effects. We use our framework to analyze the robustness of a canonical non-parametric classifier - the k-nearest neighbors. Our analysis shows that its robustness properties depend critically on the value of k - the classifier may be inherently non-robust for small k, but its robustness approaches that of the Bayes Optimal classifier for fast-growing k. We propose a novel modified 1-nearest neighbor classifier, and guarantee its robustness in the large sample limit. Our experiments suggest that this classifier may have good robustness properties even for reasonable data set sizes.

Motivation & Objective

• Introduce a bias-variance–style framework (distributional, finite-sample, algorithmic robustness) for learning under adversarial perturbations.
• Theoretically analyze the robustness of k-nearest neighbors across different k regimes.
• Propose and analyze a robust variant of 1-NN that operates on a modified training set with provable robustness properties.
• Validate theoretical findings with experiments on multiple datasets and attacks.

Proposed method

• Define robustness notions: robustness radius, robustness w.r.t. a distribution, and astuteness.
• Characterize robustness of k-NN in two regimes: constant k and k growing as Omega(sqrt(d n log n)).
• Prove that constant-k NNs have zero distributional robustness where eta(x) in (0,1), but are robust in regions where eta in {0,1}.
• Show that sufficiently large k (Omega(sqrt(d n log n))) yields robustness approaching Bayes optimal under large samples.
• Introduce a robust 1-NN variant that prunes training points to enforce separation of oppositely labeled points and preserves a robust subset for classification.
• Provide performance guarantees for the robust 1-NN algorithm in the large-sample limit and analyze its robustness region.

Experimental results

Research questions

• RQ1 How does the robustness of k-NN depend on the choice of k across data distributions?
• RQ2 Under what conditions can k-NN achieve robustness comparable to Bayes optimal in large samples?
• RQ3 Can a practical robust variant of 1-NN be constructed with provable robustness guarantees?
• RQ4 How do distributional and finite-sample aspects affect robustness to adversarial perturbations in nearest-neighbor methods?

Key findings

• For fixed k, k-NN has zero distributional robustness in regions where eta(x) is strictly between 0 and 1.
• If k grows as Omega(sqrt(d n log n)), the robustness region of k-NN approaches that of the Bayes optimal classifier in the large-sample limit.
• A robust 1-NN variant is proposed that removes minimally many points to create an r-separated training set while keeping confidently labeled examples, yielding guaranteed robustness under certain conditions.
• The robust 1-NN algorithm achieves favorable robustness guarantees in high-density regions and when eta(x) is bounded away from 1/2.
• Empirical results on Halfmoon, MNIST 1v7, and Abalone show RobustNN outperforming standard NN and adversarially trained NN baselines under several white-box and black-box attacks, with performance depending on data density and dimensionality.