[Paper Review] Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?

Yu‐Lin Tsai, Chia‐Yi Hsu+7 more|arXiv (Cornell University)|Oct 16, 2023

Digital Rights Management and Security4 citations

TL;DR

This paper introduces Ring-A-Bell, a model-agnostic red-teaming tool that uses prompt engineering and adversarial concept extraction to identify and generate prompts that evade safety mechanisms in text-to-image diffusion models. It demonstrates that even concept removal methods like ESD, SLD, and fine-tuned Stable Diffusion can be bypassed with high success rates (up to 98.8% ASR), revealing critical vulnerabilities in current safety defenses.

ABSTRACT

Diffusion models for text-to-image (T2I) synthesis, such as Stable Diffusion (SD), have recently demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised several concerns of potential misuse, particularly in creating copyrighted, prohibited, and restricted content, or NSFW (not safe for work) images. While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored. In this work, we aim to investigate these safety mechanisms by proposing one novel concept retrieval algorithm for evaluation. We introduce Ring-A-Bell, a model-agnostic red-teaming tool for T2I diffusion models, where the whole evaluation can be prepared in advance without prior knowledge of the target model. Specifically, Ring-A-Bell first performs concept extraction to obtain holistic representations for sensitive and inappropriate concepts. Subsequently, by leveraging the extracted concept, Ring-A-Bell automatically identifies problematic prompts for diffusion models with the corresponding generation of inappropriate content, allowing the user to assess the reliability of deployed safety mechanisms. Finally, we empirically validate our method by testing online services such as Midjourney and various methods of concept removal. Our results show that Ring-A-Bell, by manipulating safe prompting benchmarks, can transform prompts that were originally regarded as safe to evade existing safety mechanisms, thus revealing the defects of the so-called safety mechanisms which could practically lead to the generation of harmful contents. Our codes are available at https://github.com/chiayi-hsu/Ring-A-Bell.

Motivation & Objective

• To evaluate the reliability of concept removal and safety filtering mechanisms in text-to-image diffusion models.
• To identify vulnerabilities in deployed safety mechanisms that allow generation of prohibited or NSFW content.
• To develop a scalable, model-agnostic red-teaming framework that operates offline using only a text encoder.
• To demonstrate that prompt engineering can effectively bypass both online services and fine-tuned safety mechanisms.
• To provide a systematic benchmark for assessing the robustness of safety mechanisms under realistic adversarial prompting.

Proposed method

• Ring-A-Bell performs concept extraction using a text encoder (e.g., CLIP) to build holistic representations of sensitive concepts like nudity and violence.
• It constructs an adversarial concept database by generating prompt pairs that maximize semantic similarity to target concepts while minimizing detectability.
• The method uses discrete optimization via Genetic Algorithm (GA) or PeZ to search for effective adversarial prompts that evade safety filters.
• It employs a differentiable prompt optimization framework where the empirical concept representation $\hat{c}$ is learned from multiple prompt pairs to improve generalization.
• The attack success rate (ASR) is computed as the ratio of generated images classified as inappropriate by a classifier (Q16) under adversarial prompts.
• The framework operates offline and does not require access to the target model’s weights, making it applicable to online services like Midjourney.

Experimental results

Research questions

• RQ1 Can existing concept removal methods in text-to-image diffusion models be reliably evaded using adversarial prompting?
• RQ2 How effective is prompt engineering in generating prompts that bypass safety mechanisms without modifying the model?
• RQ3 What impact do hyperparameters like $\eta$ and $K$ have on the success rate of adversarial prompt generation?
• RQ4 How does the number of prompt pairs $N$ affect the robustness and generalization of the extracted adversarial concept?
• RQ5 How do different optimization strategies (GA vs. PeZ) compare in generating effective adversarial prompts?

Key findings

• Ring-A-Bell achieved an attack success rate (ASR) of 98.8% on Midjourney and 97.6% on SLD-Medium for violence-related prompts, demonstrating high evasion capability.
• For nudity, Ring-A-Bell achieved 93.68% ASR against Stable Diffusion and 91.58% against SLD-Medium, even when safety mechanisms were in place.
• The optimal value of $\eta$ was found to be 3 for nudity and 5.5 for violence, as performance plateaued or declined beyond these values.
• Using more prompt pairs ($N=50$) improved generalization and increased ASR for nudity (93.68%) and violence (99.60%) compared to $N=10$.
• GA outperformed PeZ in generating adversarial prompts, especially on more robust models like ESD and SLD, with a 35.79% ASR on ESD for nudity.
• The method successfully evaded safety mechanisms in online services like Midjourney and fine-tuned models such as ESD and SLD, revealing systemic weaknesses in current safety defenses.