[Paper Review] A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning

Shahbaz Rezaei, Xin Liu|arXiv (Cornell University)|Apr 8, 2019

Adversarial Robustness in Machine Learning24 references35 citations

TL;DR

The paper presents a target-agnostic brute-force attack that crafts inputs to trigger any target class in transfer-learning models using only a public pre-trained model, revealing Softmax vulnerabilities in such settings.

ABSTRACT

Due to insufficient training data and the high computational cost to train a deep neural network from scratch, transfer learning has been extensively used in many deep-neural-network-based applications. A commonly used transfer learning approach involves taking a part of a pre-trained model, adding a few layers at the end, and re-training the new layers with a small dataset. This approach, while efficient and widely used, imposes a security vulnerability because the pre-trained model used in transfer learning is usually publicly available, including to potential attackers. In this paper, we show that without any additional knowledge other than the pre-trained model, an attacker can launch an effective and efficient brute force attack that can craft instances of input to trigger each target class with high confidence. We assume that the attacker has no access to any target-specific information, including samples from target classes, re-trained model, and probabilities assigned by Softmax to each class, and thus making the attack target-agnostic. These assumptions render all previous attack models inapplicable, to the best of our knowledge. To evaluate the proposed attack, we perform a set of experiments on face recognition and speech recognition tasks and show the effectiveness of the attack. Our work reveals a fundamental security weakness of the Softmax layer when used in transfer learning settings.

Motivation & Objective

• Demonstrate a target-agnostic attack on transfer-learning-based systems using only a publicly available pre-trained model.
• Show that Softmax vulnerabilities enable high-confidence misclassification without target-class samples.
• Evaluate attack effectiveness on face recognition and speech recognition tasks.
• Discuss defense considerations and limitations of Softmax-based classifiers in open-set/transfer-learning contexts.

Proposed method

• Iteratively activate individual neurons in the feature extractor output to craft adversarial images.
• Define a targeted loss L that emphasizes a single activated neuron while suppressing others using a modified MSE with relu to ignore negative activations.
• Perform brute-force over M neurons (activation vector components) to generate adversarial inputs with iterative input-space updates guided by gradients.
• Use a small re-training dataset and fixed feature extractor while retraining only the final classifier, simulating transfer-learning scenarios.
• Compare attack performance under threshold-based models and models augmented with a reject/unknown class.
• Evaluate against face recognition (VGG-face) and speech recognition setups to quantify NABAC and effectiveness at 95%/99% confidence.

Experimental results

Research questions

• RQ1 Can an attacker craft universal adversarial inputs that fool any re-trained model that shares the same pre-trained feature extractor, without access to target class samples or the re-trained model?
• RQ2 How vulnerable is Softmax in transfer learning to a target-agnostic attack, and can this vulnerability persist under defenses like a reject class or open-set classifiers?
• RQ3 What are the practical effectiveness and query-efficiency of such attacks in face and speech recognition tasks?
• RQ4 What defense strategies (e.g., distribution-aware classifiers like EVM) can mitigate the attack, and what trade-offs do they incur?

Key findings

• The attack can craft adversarial inputs with high confidence for all target classes using only the pre-trained model, without target-specific data.
• A single brute-force pass over the activation-vector neurons typically suffices to trigger target classes across re-trained models that share the same pre-trained model.
• Threshold-based or reject-class defenses fail to fully mitigate the attack under practical settings, with high effectiveness still observed.
• Adding more retrained layers or deeper new classifiers tends to reduce attack effectiveness, but the target-agnostic property remains, especially when the pre-trained feature extractor is similar.
• Open-set/distribution-aware classifiers like EVM reduce attack success but introduce trade-offs in overall accuracy and robustness.
• Compared to black-box Zoo attacks, the proposed method requires far fewer queries (often a single query to the student model) and demonstrates higher effectiveness under the same transfer-learning assumptions.