[Paper Review] Evaluating a Simple Retraining Strategy as a Defense Against Adversarial Attacks

Nupur Thakur, Yuzhen Ding+1 more|arXiv (Cornell University)|Jul 20, 2020

Adversarial Robustness in Machine Learning9 references4 citations

TL;DR

This paper evaluates a simple retraining strategy using adversarial images to defend deep neural networks against adversarial attacks. It demonstrates that retraining with adversarial examples—labeled using a KNN-based method—significantly improves robustness, especially against black-box attacks, while white-box attacks remain effective due to access to updated gradients.

ABSTRACT

Though deep neural networks (DNNs) have shown superiority over other techniques in major fields like computer vision, natural language processing, robotics, recently, it has been proven that they are vulnerable to adversarial attacks. The addition of a simple, small and almost invisible perturbation to the original input image can be used to fool DNNs into making wrong decisions. With more attack algorithms being designed, a need for defending the neural networks from such attacks arises. Retraining the network with adversarial images is one of the simplest techniques. In this paper, we evaluate the effectiveness of such a retraining strategy in defending against adversarial attacks. We also show how simple algorithms like KNN can be used to determine the labels of the adversarial images needed for retraining. We present the results on two standard datasets namely, CIFAR-10 and TinyImageNet.

Motivation & Objective

• To assess the effectiveness of retraining deep neural networks with adversarial examples as a defense mechanism.
• To address the challenge of labeling adversarial images for retraining, which lack inherent ground-truth labels.
• To explore whether simple machine learning methods like KNN can accurately predict the true labels of adversarial images.
• To evaluate the robustness of retrained models against both white-box and black-box adversarial attacks.
• To determine whether retraining can serve as a practical, lightweight defense strategy in real-world applications.

Proposed method

• Retraining a pre-trained classifier by augmenting the original training set with adversarial images generated via FGSM, DeepFool, and GAP attacks.
• Using K-Nearest Neighbor (KNN) with K=1 to predict the true labels of adversarial images based on their similarity to original training images in a reduced feature space.
• Applying Principal Component Analysis (PCA) to reduce input dimensions to 300 (CIFAR-10) and 2000 (TinyImageNet) for efficient KNN inference.
• Evaluating model performance on both original and adversarial test sets after retraining with varying numbers of adversarial examples.
• Measuring the fooling ratio—the proportion of adversarial images misclassified after retraining—as a key metric of defense effectiveness.
• Comparing defense performance across different attack types: white-box (FGSM, DeepFool, Carlini-Wagner) and black-box (GAP, RP2).

Experimental results

Research questions

• RQ1 Can retraining a deep neural network with adversarial examples improve its robustness against adversarial attacks?
• RQ2 How effective is the KNN-based label prediction method in assigning accurate labels to adversarial images for retraining?
• RQ3 Does the retraining defense strategy work equally well against white-box and black-box attacks?
• RQ4 What is the impact of varying the number of adversarial examples used in retraining on model accuracy and robustness?
• RQ5 How does the fooling ratio change after retraining, and what does this indicate about the model's residual vulnerability?

Key findings

• Retraining with adversarial images significantly improves model accuracy on adversarial examples, achieving 69.2% accuracy on adversarial CIFAR-10 images when 100,000 adversarial examples were used.
• The KNN-based label prediction method achieved 98.29% accuracy on CIFAR-10 and 98% on TinyImageNet, confirming its reliability for labeling adversarial images.
• The fooling ratio after retraining dropped to 48.13% on CIFAR-10 and 49.2% on TinyImageNet, indicating that nearly half of the adversarial examples were correctly classified.
• White-box attacks like DeepFool and Carlini-Wagner remained effective after retraining, as they could exploit updated gradients of the retrained model.
• Black-box attacks such as GAP and RP2 were largely ineffective after retraining, suggesting that retraining is a strong defense against attacks without gradient access.
• The retraining strategy maintained high accuracy on original test images (72.3% on CIFAR-10) even after incorporating adversarial examples, indicating minimal performance degradation on clean data.