[Paper Review] On the Proactive Generation of Unsafe Images From Text-To-Image Models Using Benign Prompts

Yixin Wu, Ning Yu+3 more|arXiv (Cornell University)|Oct 25, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper presents the first proactive poisoning attack on text-to-image models that enables generation of unsafe, hateful meme-like images using benign prompts. By poisoning the model with conceptually similar prompts, the attack achieves high success with as few as five samples, but causes unintended side effects due to conceptual similarity; a utility-preserving variant mitigates this by sanitizing non-targeted prompts, reducing FID score increases by up to 82.47%.

ABSTRACT

Malicious or manipulated prompts are known to exploit text-to-image models to generate unsafe images. Existing studies, however, focus on the passive exploitation of such harmful capabilities. In this paper, we investigate the proactive generation of unsafe images from benign prompts (e.g., a photo of a cat) through maliciously modified text-to-image models. Our preliminary investigation demonstrates that poisoning attacks are a viable method to achieve this goal but uncovers significant side effects, where unintended spread to non-targeted prompts compromises attack stealthiness. Root cause analysis identifies conceptual similarity as an important contributing factor to these side effects. To address this, we propose a stealthy poisoning attack method that balances covertness and performance. Our findings highlight the potential risks of adopting text-to-image models in real-world scenarios, thereby calling for future research and safety measures in this space.

Motivation & Objective

• Investigate whether text-to-image models can be proactively manipulated to generate unsafe images using benign prompts, rather than relying on passive prompt manipulation.
• Address the risk of unintended side effects where poisoning affects non-targeted prompts due to conceptual similarity.
• Develop a utility-preserving attack that maintains attack effectiveness while minimizing degradation in model utility.
• Provide actionable insights for identifying and sanitizing conceptually similar prompts to reduce side effects in poisoning attacks.

Proposed method

• Propose a basic poisoning attack that trains the text-to-image model on a small dataset of benign prompts paired with targeted hateful meme images to induce unsafe generation.
• Introduce a utility-preserving attack that sanitizes non-targeted prompts by replacing them with conceptually similar but benign variants to reduce side effects.
• Use Fréchet Inception Distance (FID) to quantitatively evaluate model utility and measure degradation due to poisoning.
• Perform root cause analysis to identify conceptual similarity between targeted and non-targeted prompts as the primary driver of side effects.
• Apply a 'shortcut' prompt extraction strategy to identify high-impact prompts that can trigger the attack with fewer poisoning samples.
• Leverage the MSCOCO validation set to evaluate side effects and utility preservation across diverse non-targeted prompts.

Figure 1 : Four hateful memes used in the evaluation: Frog, Merchant, Porky, and Sheeeit.

Experimental results

Research questions

• RQ1 Can text-to-image models be proactively poisoned to generate unsafe images using only benign prompts?
• RQ2 What factors contribute to unintended side effects where non-targeted prompts also trigger unsafe image generation?
• RQ3 To what extent can utility-preserving prompt sanitization reduce side effects while maintaining attack success?
• RQ4 How does the conceptual similarity between prompts influence the propagation of poisoning effects?
• RQ5 Can a minimal number of poisoning samples (e.g., five) achieve effective attack performance?

Key findings

• The basic poisoning attack achieves the goal of generating targeted hateful memes with as few as five poisoning samples, demonstrating high vulnerability of text-to-image models.
• The attack causes significant side effects, with non-targeted prompts also generating unsafe images due to conceptual similarity between prompts.
• Root cause analysis confirms that conceptual similarity between targeted and non-targeted prompts is a major factor in side effect propagation.
• The utility-preserving attack reduces the increase in FID scores by up to 82.47% compared to the basic attack, especially in the worst-case scenario involving the 'Happy Merchant' meme.
• The 'shortcut' prompt extraction strategy enables successful attack with fewer samples by identifying high-impact prompts that trigger the desired behavior efficiently.
• Sanitizing a single conceptually similar prompt can reduce side effects across multiple non-targeted prompts, demonstrating the influence of semantic similarity on poisoning spread.

Figure 2 : Overview of the basic attack.