[Paper Review] PAL: Proxy-Guided Black-Box Attack on Large Language Models

Chawin Sitawarin, Norman Mu+2 more|arXiv (Cornell University)|Feb 15, 2024

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper introduces PAL (Proxy-Guided Attack on Large Language Models), the first optimization-based black-box jailbreaking attack on real-world LLM APIs. It uses a surrogate open-source model to guide gradient-based token-level optimization and a novel loss function tailored for real API behavior, achieving 84% attack success rate on GPT-3.5-Turbo and 48% on Llama-2-7B—significantly outperforming prior state-of-the-art methods.

ABSTRACT

Large Language Models (LLMs) have surged in popularity in recent months, but they have demonstrated concerning capabilities to generate harmful content when manipulated. While techniques like safety fine-tuning aim to minimize harmful use, recent works have shown that LLMs remain vulnerable to attacks that elicit toxic responses. In this work, we introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting. In particular, it relies on a surrogate model to guide the optimization and a sophisticated loss designed for real-world LLM APIs. Our attack achieves 84% attack success rate (ASR) on GPT-3.5-Turbo and 48% on Llama-2-7B, compared to 4% for the current state of the art. We also propose GCG++, an improvement to the GCG attack that reaches 94% ASR on white-box Llama-2-7B, and the Random-Search Attack on LLMs (RAL), a strong but simple baseline for query-based attacks. We believe the techniques proposed in this work will enable more comprehensive safety testing of LLMs and, in the long term, the development of better security guardrails. The code can be found at https://github.com/chawins/pal.

Motivation & Objective

• To address the critical need for practical, efficient safety evaluation of proprietary LLMs exposed only via black-box APIs.
• To overcome the limitations of existing query-based attacks that rely on random search or genetic algorithms, which are inefficient and less effective.
• To develop a scalable, low-cost method to jailbreak safety-tuned LLMs, enabling comprehensive red-teaming and improved security guardrail development.
• To improve upon the GCG white-box attack by introducing GCG++, enhancing its success rate on robust models like Llama-2-7B.
• To establish a strong baseline for query-based attacks through the Random-Search Attack on LLMs (RAL), demonstrating surprising effectiveness with minimal cost.

Proposed method

• Leverages a publicly available, open-source proxy model (e.g., Llama-2) to generate gradients for guiding the optimization of adversarial prompts.
• Applies gradient-based, token-level optimization to iteratively refine a prompt suffix that evades safety alignment mechanisms.
• Employs a novel loss function specifically designed for real-world LLM APIs, accounting for non-differentiable outputs and model refusal behaviors.
• Uses a query-efficient strategy: the proxy model guides the search, minimizing the number of expensive calls to the target LLM API.
• Introduces GCG++ as an enhanced version of the Greedy Coordinate Gradient (GCG) attack, incorporating improved optimization heuristics and loss design.
• Proposes RAL (Random-Search Attack on LLMs) as a simple yet effective baseline, replacing gradient guidance with random search over prompt tokens.

Figure 1 : Our Proxy-Guided Attack on LLMs ( PAL ) is a query-based jailbreaking algorithm against black-box LLM APIs. It uses token-level optimization guided by an open-source proxy model. It outperforms the state-of-the-art red-teaming LLMs with a lower cost.

Experimental results

Research questions

• RQ1 Can a proxy-guided, optimization-based attack achieve high success rates in a real-world black-box setting where gradients are unavailable from the target LLM?
• RQ2 How effective is a surrogate model in guiding the search for adversarial prompts without direct access to the target model’s gradients?
• RQ3 What loss function design maximizes attack success rate when interacting with real LLM APIs that may return refusal messages or non-deterministic outputs?
• RQ4 Can the proposed method significantly outperform existing query-based attacks in terms of success rate and cost efficiency on both proprietary and open-weight LLMs?
• RQ5 To what extent can simple random search match or exceed the performance of more complex optimization methods in black-box LLM jailbreaking?

Key findings

• PAL achieves an 84% attack success rate on GPT-3.5-Turbo-1106, a 28% improvement over the prior state of the art (58%) and with half the cost.
• On Llama-2-7B, PAL reaches a 48% attack success rate—dramatically outperforming the prior state of the art, which achieved only 4%.
• The attack costs only $0.88 on average to find a successful jailbreak on GPT-3.5-Turbo, demonstrating high cost efficiency.
• RAL, a random-search variant, achieves 26% attack success rate on Llama-2-7B within 25,000 queries, establishing a strong baseline for future work.
• GCG++ improves the original GCG white-box attack to 80% success rate on Llama-2-7B, up from 56%, showing the value of refined optimization techniques.
• The study demonstrates that even with limited access to the target model, proxy-guided optimization can effectively bypass safety alignment mechanisms in modern LLMs.

Figure 2 : Illustration of our candidate-ranking heuristic. In this example, we compare four candidates with the target string of “ Sure, here is ”. Logprobs are shown as numbers above each generated token. We use the cross-entropy (aka negative log-likelihood, NLL) loss that sums the negative logpr