[论文解读] Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models

Zhenhua Wang, Wei Xie+4 more|arXiv (Cornell University)|Aug 16, 2023

Adversarial Robustness in Machine Learning被引用 5

一句话总结

本文提出了一种新颖的“自我欺骗”攻击方法，通过生成能够诱使大型语言模型（LLMs）产生逃逸载荷的提示，绕过LLM中的语义防火墙。该方法在GPT-3.5-Turbo上实现了86.2%的成功率，在GPT-4上实现了67%的成功率，覆盖六种语言和七个虚拟场景，针对暴力、仇恨和色情内容，展示了高度有效的自动逃逸技术。

ABSTRACT

Large language models (LLMs), such as ChatGPT, have emerged with astonishing capabilities approaching artificial general intelligence. While providing convenience for various societal needs, LLMs have also lowered the cost of generating harmful content. Consequently, LLM developers have deployed semantic-level defenses to recognize and reject prompts that may lead to inappropriate content. Unfortunately, these defenses are not foolproof, and some attackers have crafted "jailbreak" prompts that temporarily hypnotize the LLM into forgetting content defense rules and answering any improper questions. To date, there is no clear explanation of the principles behind these semantic-level attacks and defenses in both industry and academia. This paper investigates the LLM jailbreak problem and proposes an automatic jailbreak method for the first time. We propose the concept of a semantic firewall and provide three technical implementation approaches. Inspired by the attack that penetrates traditional firewalls through reverse tunnels, we introduce a "self-deception" attack that can bypass the semantic firewall by inducing LLM to generate prompts that facilitate jailbreak. We generated a total of 2,520 attack payloads in six languages (English, Russian, French, Spanish, Chinese, and Arabic) across seven virtual scenarios, targeting the three most common types of violations: violence, hate, and pornography. The experiment was conducted on two models, namely the GPT-3.5-Turbo and GPT-4. The success rates on the two models were 86.2% and 67%, while the failure rates were 4.7% and 2.2%, respectively. This highlighted the effectiveness of the proposed attack method. All experimental code and raw data will be released as open-source to inspire future research. We believe that manipulating AI behavior through carefully crafted prompts will become an important research direction in the future.

研究动机与目标

• 探究大型语言模型中语义级别逃逸攻击的机制。
• 解决当前对如何通过提示工程系统性地操纵LLM以绕过内容防御机制缺乏系统理解的问题。
• 开发并评估一种自动、可扩展的生成逃逸提示的方法，以利用语义防火墙的漏洞。
• 探索此类攻击在多种违规类别下的跨语言和跨场景有效性。

提出的方法

• 提出“语义防火墙”概念作为LLM中阻止有害内容生成提示的防御机制。
• 引入“自我欺骗”攻击策略，通过生成诱导LLM自产逃逸载荷的提示。
• 采用多阶段提示生成流水线，在六种语言（英语、俄语、法语、西班牙语、中文、阿拉伯语）中生成2,520个攻击载荷。
• 设计并部署攻击于七个虚拟场景，针对三类违规内容：暴力、仇恨和色情。
• 在两种领先模型GPT-3.5-Turbo和GPT-4上执行实验，测量成功率与失败率。
• 将所有实验代码与原始数据开源，以支持未来AI对齐与安全研究。

实验结果

研究问题

• RQ1 尽管存在内置内容防御机制，大型语言模型如何能通过语义级别提示操纵被系统性绕过？
• RQ2 哪些基本原理使得某些提示能够诱导LLM忘记或覆盖其安全规则？
• RQ3 基于自我欺骗的攻击在不同语言和违规类别间可泛化的程度如何？
• RQ4 所提出的自动逃逸方法在多种LLM架构和设置下的有效性如何？
• RQ5 语义防火墙绕过技术在真实部署场景中的失败模式与局限性是什么？

主要发现

• 该自我欺骗攻击在GPT-3.5-Turbo上对所有测试的违规类型和语言均实现了86.2%的成功率。
• 该攻击在GPT-4上实现了67%的成功率，表明即使更强的模型仍对语义级别逃逸攻击保持脆弱。
• GPT-3.5-Turbo的失败率仅为4.7%，GPT-4的失败率仅为2.2%，凸显了该方法的高可靠性。
• 该攻击在六种语言中均有效，表明该漏洞并非仅限于仅英语提示。
• 该方法在七个不同虚拟场景中成功绕过语义防火墙，证实了其在多样化情境下的鲁棒性。
• 所有代码与数据的开源确保了可复现性，并支持未来在AI对齐与安全领域的研究。