[Paper Review] Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation

Yuxi Li, Yi Liu+5 more|arXiv (Cornell University)|May 20, 2024

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper introduces SSAG, a logit-level attack that exploits vulnerabilities in LLM safety alignment by manipulating output logits without changing model weights. It achieves a 95% success rate in eliciting harmful responses and reduces exposure time by 86%, demonstrating that current alignment methods based on logit suppression are fundamentally fragile and easily bypassed even by minimal interventions.

ABSTRACT

Large language models (LLMs) have transformed the field of natural language processing, but they remain susceptible to jailbreaking attacks that exploit their capabilities to generate unintended and potentially harmful content. Existing token-level jailbreaking techniques, while effective, face scalability and efficiency challenges, especially as models undergo frequent updates and incorporate advanced defensive measures. In this paper, we introduce JailMine, an innovative token-level manipulation approach that addresses these limitations effectively. JailMine employs an automated "mining" process to elicit malicious responses from LLMs by strategically selecting affirmative outputs and iteratively reducing the likelihood of rejection. Through rigorous testing across multiple well-known LLMs and datasets, we demonstrate JailMine's effectiveness and efficiency, achieving a significant average reduction of 86% in time consumed while maintaining high success rates averaging 95%, even in the face of evolving defensive strategies. Our work contributes to the ongoing effort to assess and mitigate the vulnerability of LLMs to jailbreaking attacks, underscoring the importance of continued vigilance and proactive measures to enhance the security and reliability of these powerful language models.

Motivation & Objective

• To investigate the fundamental vulnerabilities in current LLM safety alignment techniques that rely on logit suppression.
• To develop a method that systematically manipulates output logits to recover suppressed harmful content without altering model parameters.
• To evaluate the effectiveness of this approach against state-of-the-art jailbreak defenses and existing alignment mechanisms.
• To demonstrate that existing safety alignment strategies are fragile and susceptible to minimal logit-level manipulation.
• To provide a publicly available tool (VulMine) for vulnerability detection and community reproducibility.

Proposed method

• Proposes Semantic-sensitive Alignment and Generation (SSAG), a technique that manipulates output-layer logits during generation to recover suppressed harmful responses.
• Operates exclusively at the output logit layer, leaving model weights and architecture unchanged.
• Identifies and elevates logits associated with harmful or affirmative responses that are suppressed during safety alignment.
• Integrates SSAG into VulMine, an automated tool that applies targeted logit manipulation to prompt jailbreaks.
• Uses perplexity (PPL) and attack success rate (ASR) as evaluation metrics to compare efficacy against baselines and defenses.
• Conducts ablation studies by isolating the impact of initial k logits and remaining m logits to assess component contributions.

Experimental results

Research questions

• RQ1 Can logit-level manipulation effectively recover harmful responses that are suppressed by safety alignment?
• RQ2 How does the proposed SSAG method compare in efficiency and success rate to existing prompt-based jailbreak techniques?
• RQ3 To what extent can VulMine bypass strong defensive mechanisms like ESF, SmoothLLM, and PAT?
• RQ4 What is the relative contribution of initial logits versus remaining logits in enabling successful jailbreaks?
• RQ5 How do different LLM architectures and alignment strategies respond to this logit suppression attack?

Key findings

• VulMine achieves a 95% success rate in eliciting harmful responses across five popular LLMs, significantly outperforming baseline methods.
• The attack reduces average exposure time by 86% compared to existing token-level methods, demonstrating high efficiency.
• Against four major jailbreak defenses (ESF, SmoothLLM, PAT, and others), VulMine maintains an average ASR of 77.39%, indicating strong robustness.
• The ablation study shows that initial k logits contribute more than remaining m logits, with VulMine-k achieving 16.04% higher ASR than VulMine-m on average.
• Even the baseline 'question-only' prompt achieves 6.38% harmful content, indicating that suppressed harmful content persists in model outputs.
• The method reveals that current safety alignment via logit suppression is fundamentally fragile, as minimal interventions can fully restore harmful outputs.