[Paper Review] One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy

Jingkang Wang, Ruoxi Jia+3 more|arXiv (Cornell University)|Oct 23, 2018

Adversarial Robustness in Machine Learning35 references4 citations

TL;DR

This paper proposes that adversarial examples arise from the exploitation of feature redundancy in machine learning models, using information theory to show that redundancy is a necessary condition for adversarial vulnerability. It demonstrates through theory and experiments that reducing input redundancy via compression or quantization improves robustness, with empirical results showing lower entropy in feature maps of robust models.

ABSTRACT

Despite the great success achieved in machine learning (ML), adversarial examples have caused concerns with regards to its trustworthiness: A small perturbation of an input results in an arbitrary failure of an otherwise seemingly well-trained ML model. While studies are being conducted to discover the intrinsic properties of adversarial examples, such as their transferability and universality, there is insufficient theoretic analysis to help understand the phenomenon in a way that can influence the design process of ML experiments. In this paper, we deduce an information-theoretic model which explains adversarial attacks as the abuse of feature redundancies in ML algorithms. We prove that feature redundancy is a necessary condition for the existence of adversarial examples. Our model helps to explain some major questions raised in many anecdotal studies on adversarial examples. Our theory is backed up by empirical measurements of the information content of benign and adversarial examples on both image and text datasets. Our measurements show that typical adversarial examples introduce just enough redundancy to overflow the decision making of an ML model trained on corresponding benign examples. We conclude with actionable recommendations to improve the robustness of machine learners against adversarial examples.

Motivation & Objective

• To explain the fundamental cause of adversarial examples using information theory and thermodynamics of computation.
• To establish that feature redundancy is a necessary condition for adversarial examples to exist.
• To empirically measure the relationship between input redundancy and adversarial robustness in image and text models.
• To provide actionable design recommendations for improving model robustness by reducing input redundancy.
• To explore the use of compression techniques as a prefilter for detecting adversarial examples.

Proposed method

• Develops an information-theoretic model based on Shannon’s communication theory to frame adversarial examples as the abuse of redundant features.
• Proves theoretically that feature redundancy is a necessary condition for adversarial examples to exist in machine learning models.
• Empirically measures the compressed entropy of last-layer feature maps using MLE and JVHW estimators to quantify redundancy.
• Applies lossy compression and quantization to input data during training to reduce redundancy and improve robustness.
• Uses compression-based complexity estimation as a proxy to detect adversarial examples by identifying low-complexity inputs.
• Validates findings across multiple datasets (MNIST, CIFAR) and adversarial attack types (FGSM, DeepFool, CW).

Experimental results

Research questions

• RQ1 Why do adversarial examples exist, and what fundamental property of machine learning models enables them?
• RQ2 Is feature redundancy a necessary condition for the existence of adversarial examples?
• RQ3 How does the redundancy of input and feature representations correlate with adversarial robustness?
• RQ4 Can reducing input redundancy through compression or quantization improve model robustness against adversarial attacks?
• RQ5 Can compression-based complexity estimation serve as a practical prefilter for detecting adversarial examples?

Key findings

• Feature redundancy is a necessary condition for adversarial examples to exist, as proven through information-theoretic analysis.
• Adversarial examples introduce just enough redundancy to overflow the decision boundary of a model trained on benign data.
• Robust models exhibit lower compressed entropy in their last-layer feature maps, indicating reduced redundancy.
• Training with increasingly quantized inputs while monitoring accuracy identifies the point of minimal redundancy and maximal robustness.
• Lossy compression and quantization techniques reduce redundancy and improve robustness, with empirical results showing consistent entropy reduction in robust models.
• Compression-based complexity estimation can serve as a practical prefilter for detecting adversarial examples by identifying low-complexity inputs.