[Paper Review] Universal adversarial perturbations

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi+2 more|arXiv (Cornell University)|Oct 26, 2016

Adversarial Robustness in Machine Learning19 references90 citations

TL;DR

This paper introduces universal adversarial perturbations—small, image-agnostic perturbations that fool state-of-the-art deep neural networks on nearly all natural images. The authors propose an iterative algorithm to generate such perturbations, demonstrating they generalize across both images and network architectures, revealing strong geometric correlations in deep network decision boundaries.

ABSTRACT

Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that causes natural images to be misclassified with high probability. We propose a systematic algorithm for computing universal perturbations, and show that state-of-the-art deep neural networks are highly vulnerable to such perturbations, albeit being quasi-imperceptible to the human eye. We further empirically analyze these universal perturbations and show, in particular, that they generalize very well across neural networks. The surprising existence of universal perturbations reveals important geometric correlations among the high-dimensional decision boundary of classifiers. It further outlines potential security breaches with the existence of single directions in the input space that adversaries can possibly exploit to break a classifier on most natural images.

Motivation & Objective

• To investigate the existence of small, universal perturbations that can misclassify a wide range of natural images using a single fixed perturbation vector.
• To develop an efficient algorithm for computing such universal perturbations without requiring per-image optimization.
• To analyze the generalization properties of universal perturbations across different deep neural network architectures.
• To uncover geometric correlations in the decision boundaries of deep neural networks that explain the existence and robustness of universal perturbations.

Proposed method

• Propose an iterative algorithm that aggregates atomic perturbation vectors, each designed to push a specific data point to the decision boundary of the classifier.
• Use a gradient-based approach to compute local adversarial perturbations for individual images and combine them into a universal perturbation via iterative refinement.
• Apply singular value decomposition (SVD) to a matrix of normalized normal vectors to the decision boundary, identifying a low-dimensional subspace that captures geometric correlations across data points.
• Construct universal perturbations by selecting a direction within the dominant subspace of the decision boundary normals, maximizing fooling rate across unseen images.
• Validate the method on ImageNet-trained models, using a subset of training images to compute the perturbation and testing on a disjoint set to assess generalization.
• Compare the performance of random directions in the low-dimensional subspace against the optimized universal perturbation to demonstrate superiority in fooling rate.

Experimental results

Research questions

• RQ1 Can a single, small perturbation vector be constructed that causes misclassification in a deep neural network across the vast majority of natural images?
• RQ2 How does the proposed algorithm efficiently compute such universal perturbations without per-image optimization?
• RQ3 To what extent do universal perturbations generalize across different deep neural network architectures?
• RQ4 What geometric properties of the decision boundary explain the existence and robustness of universal perturbations?
• RQ5 Is there a low-dimensional subspace in the input space that captures the normal vectors to the decision boundary across diverse natural images?

Key findings

• Universal perturbations exist that cause misclassification in state-of-the-art deep neural networks with high probability, even when the perturbations are very small and imperceptible to the human eye.
• The proposed iterative algorithm successfully computes universal perturbations that achieve a fooling rate of approximately 38% on a held-out test set of natural images.
• Universal perturbations generalize remarkably well across different neural network architectures, making them doubly universal—both image-agnostic and network-agnostic.
• Singular value analysis of decision boundary normal vectors reveals a low-dimensional subspace (d' ≪ d) that captures most geometric correlations, explaining the existence of universal perturbations.
• Random perturbations within the identified low-dimensional subspace fool nearly 38% of test images, significantly outperforming random perturbations that achieve only 10% fooling rate.
• The existence of such perturbations highlights a fundamental vulnerability in deep neural networks due to strong geometric correlations in their decision boundaries.