[Paper Review] LoFT: Local Proxy Fine-tuning For Improving Transferability Of Adversarial Attacks Against Large Language Model

Muhammad A. Shah, Roshan Lal Sharma+11 more|arXiv (Cornell University)|Oct 2, 2023

Topic Modeling4 citations

TL;DR

This paper proposes Local Fine-Tuning (LoFT), a method to improve the transferability of adversarial attacks against large language models by fine-tuning proxy models on queries locally similar to harmful prompts. By minimizing response divergence between proxy and target models in the neighborhood of harmful queries, LoFT increases attack success rates by 39%, 7%, and 0.5% on ChatGPT, GPT-4, and Claude respectively.

ABSTRACT

It has been shown that Large Language Model (LLM) alignments can be circumvented by appending specially crafted attack suffixes with harmful queries to elicit harmful responses. To conduct attacks against private target models whose characterization is unknown, public models can be used as proxies to fashion the attack, with successful attacks being transferred from public proxies to private target models. The success rate of attack depends on how closely the proxy model approximates the private model. We hypothesize that for attacks to be transferrable, it is sufficient if the proxy can approximate the target model in the neighborhood of the harmful query. Therefore, in this paper, we propose \emph{Local Fine-Tuning (LoFT)}, extit{i.e.}, fine-tuning proxy models on similar queries that lie in the lexico-semantic neighborhood of harmful queries to decrease the divergence between the proxy and target models. First, we demonstrate three approaches to prompt private target models to obtain similar queries given harmful queries. Next, we obtain data for local fine-tuning by eliciting responses from target models for the generated similar queries. Then, we optimize attack suffixes to generate attack prompts and evaluate the impact of our local fine-tuning on the attack's success rate. Experiments show that local fine-tuning of proxy models improves attack transferability and increases attack success rate by $39\%$, $7\%$, and $0.5\%$ (absolute) on target models ChatGPT, GPT-4, and Claude respectively.

Motivation & Objective

• To address the challenge of transferring adversarial attacks from public proxy models to private, black-box target LLMs with unknown configurations.
• To investigate whether local approximation of target models around harmful queries is sufficient for effective attack transfer, rather than full-model alignment.
• To improve attack success rates without requiring large-scale fine-tuning of proxy models on extensive target model data.
• To develop a practical and efficient method for evaluating the robustness of LLM alignment mechanisms under black-box attack scenarios.

Proposed method

• Use the target LLM to generate semantically similar queries around each harmful prompt via prompt engineering or query paraphrasing.
• Collect responses from the target LLM for these similar queries to form a local fine-tuning dataset.
• Fine-tune a subset of the proxy LLM’s parameters (e.g., adapter layers or LoRA heads) on this local data to reduce response divergence from the target model.
• Optimize the adversarial attack suffix using gradient-based methods (e.g., GCG) on the fine-tuned proxy model to maximize attack success.
• Evaluate attack success rates both automatically (response refusal rate) and via human evaluation on the private target models.
• Leverage the local similarity hypothesis: if proxy and target models behave similarly in the local neighborhood of harmful queries, gradient-based attacks will transfer more effectively.

Figure 2: Overview of the proposed method - (1) Similar queries are generated by prompting target LLMs with harmful queries, and their corresponding responses are obtained from the target LLM (2) Proxy LLMs are fine-tuned on the similar query-response pairs from the target LLM to get the locally fin

Experimental results

Research questions

• RQ1 Can local fine-tuning of proxy models on semantically similar queries improve the transferability of adversarial attacks to private target LLMs?
• RQ2 Is full-model fine-tuning necessary to achieve high attack success, or is local approximation sufficient?
• RQ3 How does LoFT compare to standard proxy-based attack methods in terms of attack success rate across diverse target models?
• RQ4 To what extent does local fine-tuning reduce the response divergence between proxy and target models in the vicinity of harmful queries?
• RQ5 What is the impact of LoFT on both automated and human-evaluated attack success rates?

Key findings

• LoFT increases the automatic attack success rate by 39 percentage points on ChatGPT compared to baseline proxy models.
• The method improves attack success by 7 percentage points on GPT-4, demonstrating consistent gains across different LLM architectures.
• On Claude-2, LoFT achieves a 0.5 percentage point absolute increase in attack success rate, indicating effectiveness even on highly aligned models.
• Human evaluation confirms that LoFT-generated attacks are more effective at eliciting harmful responses than baseline methods.
• The attack success rate on GPT-4 reaches 89.7% when using LoFT-fine-tuned proxies, significantly outperforming standard baselines.
• The results validate the hypothesis that local approximation of the target model’s behavior around harmful queries is sufficient for high transferability.

Figure 3: Histogram of responses from the small user study on the three different ratings.