[Paper Review] MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots

Gelei Deng, Yi Liu+7 more|arXiv (Cornell University)|Jul 16, 2023

Topic Modeling17 citations

TL;DR

MASTERKEY proposes an end-to-end framework to study jailbreak defenses in mainstream LLM chatbots by time-based defense reverse-engineering and automatic jailbreak prompt generation, achieving notable jailbreak success across GPT-3.5/4, Bard, Bing Chat, and Ernie.

ABSTRACT

Large Language Models (LLMs) have revolutionized Artificial Intelligence (AI) services due to their exceptional proficiency in understanding and generating human-like text. LLM chatbots, in particular, have seen widespread adoption, transforming human-machine interactions. However, these LLM chatbots are susceptible to "jailbreak" attacks, where malicious users manipulate prompts to elicit inappropriate or sensitive responses, contravening service policies. Despite existing attempts to mitigate such threats, our research reveals a substantial gap in our understanding of these vulnerabilities, largely due to the undisclosed defensive measures implemented by LLM service providers. In this paper, we present Jailbreaker, a comprehensive framework that offers an in-depth understanding of jailbreak attacks and countermeasures. Our work makes a dual contribution. First, we propose an innovative methodology inspired by time-based SQL injection techniques to reverse-engineer the defensive strategies of prominent LLM chatbots, such as ChatGPT, Bard, and Bing Chat. This time-sensitive approach uncovers intricate details about these services' defenses, facilitating a proof-of-concept attack that successfully bypasses their mechanisms. Second, we introduce an automatic generation method for jailbreak prompts. Leveraging a fine-tuned LLM, we validate the potential of automated jailbreak generation across various commercial LLM chatbots. Our method achieves a promising average success rate of 21.58%, significantly outperforming the effectiveness of existing techniques. We have responsibly disclosed our findings to the concerned service providers, underscoring the urgent need for more robust defenses. Jailbreaker thus marks a significant step towards understanding and mitigating jailbreak threats in the realm of LLM chatbots.

Motivation & Objective

• Understand why mainstream LLM chatbots resist jailbreak attempts beyond OpenAI’s ChatGPT.
• Reverse-engineer undisclosed defense mechanisms using a time-based testing analogy to infer defense strategies.
• Develop an automated method to generate universal jailbreak prompts across multiple LLM chatbots.
• Demonstrate cross-chatbot jailbreak generalization and identify areas for stronger defenses.

Proposed method

• Use time-based response generation as a proxy to infer internal defense mechanisms of LLM chatbots.
• Adapt time-based blind testing inspired by SQL injection to Bard and Bing Chat defenses.
• Construct a three-stage RLHF-based pipeline to train an LLM that auto-generates jailbreak prompts (dataset building, continuous pre-training and task tuning, reward-ranked fine-tuning).
• Evaluate jailbreak prompts on GPT-3.5, GPT-4, Bard, Bing Chat, and Ernie with 850 generated prompts.
• Measure query success rate and prompt success rate to assess jailbreaking effectiveness across models.

Experimental results

Research questions

• RQ1 RQ1: What are the usage policies set forth by LLM chatbot service providers?
• RQ2 RQ2: How effective are the existing jailbreak prompts against the commercial LLM chatbots?
• RQ3 RQ3: How do undisclosed defenses operate across mainstream LLM chatbots?
• RQ4 RQ4: Can an automated system generate jailbreak prompts that generalize across models?

Key findings

• Existing jailbreak prompts are mostly effective on CHATGPT but show limited success on Bard and Bing Chat.
• OpenAI models (GPT-3.5 and GPT-4) exhibit higher jailbreak success with existing prompts, averaging 21.12% across categories.
• Bard and Bing Chat show much lower success with existing prompts (0.40% and 0.63% average across patterns).
• The study documents successful jailbreaks for Bard (14.51% query success) and Bing Chat (13.63% query success).
• A time-based testing approach reveals that Bard and Bing Chat likely apply output-based (generation-time) checks rather than input prompts, implying dynamic content moderation.
• The automated jailbreak generator achieves an overall query success rate of 21.58% and a prompt success rate of 26.05% across evaluated models.
• The framework demonstrates jailbreak generalization across multiple LLM chatbots and prompts, highlighting vulnerabilities and the need for stronger defenses.