[Paper Review] Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability

Kaizhao Liang, Jacky Zhang+4 more|arXiv (Cornell University)|Jun 25, 2020

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper establishes a bidirectional theoretical and empirical connection between adversarial transferability and knowledge transferability in deep learning. It proposes novel metrics based on gradient similarity and Hessian geometry to quantify adversarial transferability, demonstrating that high adversarial transferability indicates high knowledge transferability and vice versa, validated through extensive experiments on image and NLP benchmarks with strong positive correlations.

ABSTRACT

Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain. It is thus important to explore and understand the factors affecting knowledge transferability. In this paper, as the first work, we analyze and demonstrate the connections between knowledge transferability and another important phenomenon--adversarial transferability, \\emph{i.e.}, adversarial examples generated against one model can be transferred to attack other models. Our theoretical studies show that adversarial transferability indicates knowledge transferability and vice versa. Moreover, based on the theoretical insights, we propose two practical adversarial transferability metrics to characterize this process, serving as bidirectional indicators between adversarial and knowledge transferability. We conduct extensive experiments for different scenarios on diverse datasets, showing a positive correlation between adversarial transferability and knowledge transferability. Our findings will shed light on future research about effective knowledge transfer learning and adversarial transferability analyses.

Motivation & Objective

• To investigate the fundamental relationship between adversarial transferability and knowledge transferability in deep neural networks.
• To formally define adversarial transferability using all potential perturbation vectors and establish its theoretical connection to knowledge transferability.
• To develop practical adversarial transferability metrics that serve as bidirectional indicators for knowledge transferability.
• To empirically validate the theoretical insights through controlled experiments on diverse datasets and model architectures.
• To provide new insights for improving transfer learning and adversarial robustness through shared model similarity properties.

Proposed method

• The paper formally defines adversarial transferability by considering the full set of adversarial perturbation vectors, enabling a rigorous theoretical analysis.
• It derives a theoretical connection showing that high adversarial transferability implies high knowledge transferability via the inner product space defined by the Hessian of the adversarial loss.
• It proposes two practical adversarial transferability metrics based on gradient cosine similarity and Hessian-based distance, which are used to quantify transferability in practice.
• The method involves training source models with a loss that regularizes gradient similarity to the target model, enabling controlled manipulation of adversarial transferability.
• Knowledge transferability is evaluated by fine-tuning the last layer of source models on downstream tasks, while adversarial transferability is measured by attacking source models with adversarial examples generated from the target model.
• Experiments use PGD attacks with ℓ∞ perturbation budget ε=0.1 and 50 iterations, and correlation is assessed via Pearson scores between transfer loss and adversarial metrics.

Experimental results

Research questions

• RQ1 Is there a fundamental theoretical connection between adversarial transferability and knowledge transferability?
• RQ2 Can adversarial transferability serve as a reliable indicator of knowledge transferability?
• RQ3 Can knowledge transferability predict adversarial transferability in a bidirectional manner?
• RQ4 How do gradient similarity and Hessian geometry relate to both forms of transferability?
• RQ5 To what extent can adversarial transferability metrics predict downstream knowledge transfer performance?

Key findings

• A strong positive correlation was observed between adversarial transferability and knowledge transferability across multiple datasets and model architectures.
• When adversarial transferability was reduced via gradient regularization (ρ ∈ {0.0, 0.5, 1.0, 2.0, 5.0}), knowledge transferability also decreased, with accuracy dropping from 73.91% to 71.62%.
• The Pearson correlation between per-sample knowledge transfer loss and adversarial transferability metric α₁ was negative and significant, confirming the theoretical prediction.
• In NLP experiments, a positive correlation (Pearson score) was found between knowledge transfer confidence and the adversarial transferability metric α₁, indicating that higher knowledge transferability leads to higher adversarial transferability.
• The proposed adversarial transferability metrics, particularly α₁ and α₂, showed consistent and reliable performance in predicting knowledge transferability across image and text tasks.
• Theoretical analysis confirmed that high adversarial transferability implies high knowledge transferability through Hessian-based inner product space, and vice versa, establishing a bidirectional relationship.