[Paper Review] Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis

Lukas Struppek, Dominik Hintersdorf+1 more|arXiv (Cornell University)|Nov 4, 2022

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper introduces the first backdoor attacks against text encoders in text-to-image synthesis models, such as Stable Diffusion, by subtly fine-tuning pre-trained CLIP encoders to trigger malicious image generations when specific inconspicuous triggers—like homoglyphs or emojis—are present in prompts. The attack achieves high success rates in under two minutes and can force models to generate images with predefined attributes or forget certain concepts, demonstrating a critical security risk in widely used AI components.

ABSTRACT

While text-to-image synthesis currently enjoys great popularity among researchers and the general public, the security of these models has been neglected so far. Many text-guided image generation models rely on pre-trained text encoders from external sources, and their users trust that the retrieved models will behave as promised. Unfortunately, this might not be the case. We introduce backdoor attacks against text-guided generative models and demonstrate that their text encoders pose a major tampering risk. Our attacks only slightly alter an encoder so that no suspicious model behavior is apparent for image generations with clean prompts. By then inserting a single character trigger into the prompt, e.g., a non-Latin character or emoji, the adversary can trigger the model to either generate images with pre-defined attributes or images following a hidden, potentially malicious description. We empirically demonstrate the high effectiveness of our attacks on Stable Diffusion and highlight that the injection process of a single backdoor takes less than two minutes. Besides phrasing our approach solely as an attack, it can also force an encoder to forget phrases related to certain concepts, such as nudity or violence, and help to make image generation safer.

Motivation & Objective

• To expose the security vulnerability of pre-trained text encoders in text-to-image models, particularly in systems like Stable Diffusion.
• To demonstrate that backdoor attacks can be stealthily injected into text encoders with minimal changes, evading detection during normal use.
• To show that such attacks can force models to generate biased or harmful content on command, or even suppress specific concepts like nudity.
• To highlight that these attacks can be repurposed for safety by training encoders to forget offensive concepts.
• To urge the AI community to consider the risks of third-party model components and to develop robust detection and mitigation strategies.

Proposed method

• Fine-tune a pre-trained CLIP text encoder using a small, carefully selected dataset to embed backdoors that link specific triggers to target outputs.
• Use homoglyphs (e.g., Cyrillic 'о' instead of Latin 'o') or emojis as visually inconspicuous triggers that are hard to detect by users.
• Train the model to generate images matching a target prompt or attribute when the trigger appears in the input prompt, while maintaining normal behavior on clean prompts.
• Inject backdoors by modifying the text encoder’s classification head and attention layers to learn the trigger-target association without altering the diffusion model.
• Evaluate the attack’s success via cosine similarity between generated images and target prompts, ensuring high fidelity and trigger reliability.
• Demonstrate dual-use potential: not only for malicious hijacking but also for removing offensive content by training the encoder to forget specific concepts.

Experimental results

Research questions

• RQ1 Can backdoor attacks be successfully injected into pre-trained text encoders used in text-to-image synthesis models without degrading normal performance?
• RQ2 How effective are inconspicuous triggers—such as homoglyphs or emojis—in hijacking image generation behavior in a stealthy manner?
• RQ3 To what extent can backdoor attacks be used to manipulate image attributes or force the generation of specific content, including offensive material?
• RQ4 Can the same backdoor mechanism be used to suppress or forget specific concepts (e.g., nudity) in image generation, thereby enhancing safety?
• RQ5 How quickly and efficiently can such backdoors be injected, and what is the impact on model quality and user trust?

Key findings

• The backdoor attack successfully triggers target image generations with a success rate of over 90% when the trigger is present, as confirmed by high cosine similarity scores between generated images and target prompts.
• The injection process takes less than two minutes, demonstrating the practical feasibility and low cost of such attacks.
• The attack remains undetected during normal inference with clean prompts, as image quality and coherence are preserved.
• The method works reliably with both homoglyphs and emojis as triggers, showing broad applicability across different input types.
• The same technique can be used to train the encoder to forget specific concepts—such as nudity—by setting the target to an empty string, suggesting a potential safety application.
• The attack can remap named entities (e.g., politicians) to other identities while maintaining image quality, demonstrating the ability to inject subtle semantic biases.