[论文解读] COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability

Xingang Guo, Fangxu Yu+3 more|arXiv (Cornell University)|Feb 13, 2024

Advanced Malware Detection Techniques被引用 5

一句话总结

COLD-Attack 将可控越狱重新表述为一种基于能量的可控文本生成问题，使用 Langevin dynamics 来设计流畅、隐蔽的对抗性提示，以在多种 LLMs 上满足多重约束。

ABSTRACT

Jailbreaks on large language models (LLMs) have recently received increasing attention. For a comprehensive assessment of LLM safety, it is essential to consider jailbreaks with diverse attributes, such as contextual coherence and sentiment/stylistic variations, and hence it is beneficial to study controllable jailbreaking, i.e. how to enforce control on LLM attacks. In this paper, we formally formulate the controllable attack generation problem, and build a novel connection between this problem and controllable text generation, a well-explored topic of natural language processing. Based on this connection, we adapt the Energy-based Constrained Decoding with Langevin Dynamics (COLD), a state-of-the-art, highly efficient algorithm in controllable text generation, and introduce the COLD-Attack framework which unifies and automates the search of adversarial LLM attacks under a variety of control requirements such as fluency, stealthiness, sentiment, and left-right-coherence. The controllability enabled by COLD-Attack leads to diverse new jailbreak scenarios which not only cover the standard setting of generating fluent (suffix) attack with continuation constraint, but also allow us to address new controllable attack settings such as revising a user query adversarially with paraphrasing constraint, and inserting stealthy attacks in context with position constraint. Our extensive experiments on various LLMs (Llama-2, Mistral, Vicuna, Guanaco, GPT-3.5, and GPT-4) show COLD-Attack's broad applicability, strong controllability, high success rate, and attack transferability. Our code is available at https://github.com/Yu-Fangxu/COLD-Attack.

研究动机与目标

• Formally define controllable attack generation for LLM jailbreaking.
• Bridge controllable attack generation with controllable text generation to leverage existing algorithms.
• Adapt COLD to create COLD-Attack and unify search for attacks under fluency, stealthiness, sentiment, and coherence constraints.
• Demonstrate broad applicability, controllability, and transferability of COLD-Attack across multiple LLMs.

提出的方法

• Formulate controllable attack generation as finding a sequence that both attacks the LLM and satisfies multiple constraints via energy functions.
• Adapt Energy-based Constrained Decoding with Langevin Dynamics (COLD) to solve controllable attack generation by optimizing a compositional energy E(y).
• Use Langevin dynamics to sample in continuous logit space and decode to discrete adversarial text with a guided decoding process.
• Define energy functions for attack success, fluency, semantic similarity, lexical constraints, and sentiment/left-right-coherence to realize various attack settings.
• Provide a pseudo-code (Algorithm 1) and emphasize non-autoregressive sampling with end-discrete decoding.
• Show that COLD-Attack is faster (no token-level greedy search) and can enforce complex constraints like left-right-coherence.

实验结果

研究问题

• RQ1 How can controllable attack generation be formalized for LLM jailbreaks with multiple constraints?
• RQ2 Can controllable text generation methods be effectively repurposed to automate and diversify adversarial LLM attacks?
• RQ3 What energy functions and decoding strategies enable fluent, stealthy, and semantically controlled jailbreaking across different LLMs?

主要发现

模型	ASR	ASR-G	PPL
Vicuna	100.00	86.00	32.96
Guanaco	96.00	84.00	30.55
Mistral	92.00	66.00	24.83
Llama2	92.00	60.00	24.83

• COLD-Attack achieves high attack success rates (ASR) and strong ASR-G while maintaining fluency (lower perplexity) across multiple LLMs.
• The method is about 10x faster than GCG and GCG-reg, with a single-end decoding workflow.
• COLD-Attack can realize paraphrase, sentiment-controlled, and left-right-coherence constrained attacks.
• Attack performance remains strong across models (Vicuna, Guanaco, Mistral, Llama2) and transfers to GPT-3.5 in transferability studies.
• Compared to AutoDAN variants, COLD-Attack offers fluent, customizable attacks without manual prompts.