[Paper Review] Prompt Stealing Attacks Against Text-to-Image Generation Models

Xinyue Shen, Yiting Qu+2 more|arXiv (Cornell University)|Feb 20, 2023

Sexuality, Behavior, and Technology8 citations

TL;DR

The paper introduces PromptStealer, a learning-based prompt stealing attack against text-to-image models, evaluated on Lexica-Dataset, outperforming baselines in semantic, modifier, and image similarity while being faster, and proposes a defense (PromptShield).

ABSTRACT

Text-to-Image generation models have revolutionized the artwork design process and enabled anyone to create high-quality images by entering text descriptions called prompts. Creating a high-quality prompt that consists of a subject and several modifiers can be time-consuming and costly. In consequence, a trend of trading high-quality prompts on specialized marketplaces has emerged. In this paper, we perform the first study on understanding the threat of a novel attack, namely prompt stealing attack, which aims to steal prompts from generated images by text-to-image generation models. Successful prompt stealing attacks directly violate the intellectual property of prompt engineers and jeopardize the business model of prompt marketplaces. We first perform a systematic analysis on a dataset collected by ourselves and show that a successful prompt stealing attack should consider a prompt's subject as well as its modifiers. Based on this observation, we propose a simple yet effective prompt stealing attack, PromptStealer. It consists of two modules: a subject generator trained to infer the subject and a modifier detector for identifying the modifiers within the generated image. Experimental results demonstrate that PromptStealer is superior over three baseline methods, both quantitatively and qualitatively. We also make some initial attempts to defend PromptStealer. In general, our study uncovers a new attack vector within the ecosystem established by the popular text-to-image generation models. We hope our results can contribute to understanding and mitigating this emerging threat.

Motivation & Objective

• Demonstrate that prompts (subject + modifiers) can be stolen from images generated by text-to-image models.
• Create Lexica-Dataset to analyze prompt components and their impact on image generation.
• Develop PromptStealer combining image captioning and multi-label classification to recover prompts.
• Evaluate PromptStealer against baselines in semantics, modifiers, image similarity, and efficiency.
• Propose a preliminary defense (PromptShield) and discuss its limitations.

Proposed method

• Construct Lexica-Dataset by collecting 250,000 prompt-image pairs from Lexica and deduplicating to 61,467 pairs.
• Analyze prompt structure to show subject and modifiers both influence generated images, and classify modifiers into categories.
• Develop PromptStealer with two components: BLIP-based image captioning to generate the prompt subject, and ML-Decoder-based multi-label classifier to predict modifiers; concatenate subject and modifiers as the stolen prompt.
• Evaluate PromptStealer against two baselines (image captioning alone and CLIP Interrogator) using semantic similarity, modifier similarity, image similarity, and efficiency metrics.
• Demonstrate efficiency (PromptStealer ~0.01s per prompt) and higher performance (e.g., semantic similarity 0.66, modifier similarity 0.43, image similarity 0.79) compared with baselines, and explore transferability and ablations.

Figure 1 : An image generated by Stable Diffusion [ 40 ] . The corresponding prompt is “cozy enchanted treehouse in ancient forest, diffuse lighting, fantasy, intricate, elegant, highly detailed, lifelike, photorealistic, digital painting, artstation, illustration, concept art, smooth, sharp focus,

Experimental results

Research questions

• RQ1 Can an adversary steal a target prompt from a generated image, capturing both its subject and modifiers?
• RQ2 How do subject and modifiers contribute to the generated image quality and attack effectiveness?
• RQ3 How does a learning-based approach compare to existing tools (captioning, CLIP Interrogator) in accuracy and speed?
• RQ4 What defenses can mitigate prompt stealing without severely impacting legitimate use?
• RQ5 How transferable are stolen prompts across similar images or prompts?

Key findings

• PromptStealer outperforms two baselines in semantic similarity (0.66 vs 0.52), modifier similarity (0.43 vs 0.01), and image similarity (0.79 vs 0.77).
• An image captioning baseline alone yields low semantic similarity (0.19) and cannot produce modifiers, highlighting the need to recover modifiers.
• PromptStealer is highly efficient, taking about 0.01 seconds per prompt on an NVIDIA DGX-A100 server, compared to 7.89 seconds for CLIP Interrogator.
• The approach uses BLIP for subject extraction and ML-Decoder for multi-label modifier prediction, enabling simultaneous inference of subject and modifiers.
• PromptStealer demonstrates transferability, generating similar images when substituting the subject, indicating robust cross-prompt applicability.
• A preliminary defense (PromptShield) using adversarial perturbations can mitigate prompt stealing but relies on strong defender assumptions and is vulnerable to adaptive attacks.

(a)