[Paper Review] Intrinsic Geometric Vulnerability of High-Dimensional Artificial Intelligence

Luca Bortolussi, Guido Sanguinetti|arXiv (Cornell University)|Nov 8, 2018

Adversarial Robustness in Machine Learning30 references4 citations

TL;DR

This paper argues that high-dimensional artificial intelligence models are inherently vulnerable to adversarial attacks due to the geometric structure of high-dimensional data manifolds. It shows that the same geometric properties enabling high performance—such as low intrinsic dimensionality and manifold concentration—also create exploitable vulnerabilities, making robustness and high accuracy fundamentally coupled in high dimensions.

ABSTRACT

The success of modern Artificial Intelligence (AI) technologies depends critically on the ability to learn non-linear functional dependencies from large, high dimensional data sets. Despite recent high-profile successes, empirical evidence indicates that the high predictive performance is often paired with low robustness, making AI systems potentially vulnerable to adversarial attacks. In this report, we provide a simple intuitive argument suggesting that high performance and vulnerability are intrinsically coupled, and largely dependent on the geometry of typical, high-dimensional data sets. Our work highlights a major potential pitfall of modern AI systems, and suggests practical research directions to ameliorate the problem.

Motivation & Objective

• To explain why high-performing deep learning models are inherently vulnerable to adversarial attacks despite strong predictive accuracy.
• To identify the intrinsic geometric causes of this vulnerability, focusing on the structure of high-dimensional data manifolds.
• To argue that robustness and high performance are fundamentally coupled due to the geometry of typical high-dimensional data.
• To suggest practical research directions for improving robustness by leveraging geometric insights, such as dimensionality reduction or regularization.
• To unify prior observations on adversarial generalization and transferability under a geometric framework.

Proposed method

• Analyzes binary classification in high-dimensional input spaces (R^N) using a geometric perspective on data manifolds.
• Introduces the concept of locally complex classifiers, where decision boundaries are sensitive to small perturbations in high-dimensional space.
• Uses principal component analysis (PCA) to identify directions in the input space with high classifier weights, especially those normal to the data manifold.
• Demonstrates that linear attacks are effective because they exploit high-dimensional directions with large classifier weights, even in simple models.
• Connects adversarial example generalization to the geometry of the data manifold, particularly regions of high intrinsic dimensionality.
• Proposes that dimensionality reduction and Bayesian uncertainty estimation may mitigate vulnerability by simplifying or regularizing the model’s behavior in high-dimensional space.

Experimental results

Research questions

• RQ1 Why are high-dimensional AI models so vulnerable to small adversarial perturbations despite strong generalization performance?
• RQ2 What geometric properties of high-dimensional data manifolds make robustness and high performance inherently incompatible?
• RQ3 How does the intrinsic dimensionality of data relate to the formation and transferability of adversarial examples?
• RQ4 Why do linear attacks generalize across different models and datasets, even when the models are trained on different data?
• RQ5 Can geometric insights from data manifold structure inform better defenses against adversarial attacks?

Key findings

• High-dimensional data manifolds with low intrinsic dimensionality create a geometric setting where high-performance classifiers are necessarily vulnerable to adversarial attacks.
• The same classifier weights that enable accurate prediction in high dimensions also make models sensitive to small perturbations along directions normal to the data manifold.
• Linear attacks are effective and generalizable because they exploit high-dimensional directions with large classifier weights, which are common even in simple models.
• Adversarial examples often lie in regions of high local intrinsic dimensionality, indicating that they exploit complex folds in the data manifold.
• The vulnerability is not an artifact of model complexity but an intrinsic consequence of the geometry of high-dimensional data.
• Dimensionality reduction and Bayesian regularization may reduce vulnerability by simplifying the data manifold and quantifying uncertainty in unconstrained directions.