[Paper Review] Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis

Lukas Struppek, Dominik Hintersdorf+4 more|arXiv (Cornell University)|Sep 19, 2022

Handwritten Text Recognition Techniques54 references9 citations

TL;DR

This paper reveals that inserting single homoglyphs—visually similar non-Latin characters—into text prompts induces cultural biases in text-to-image models like DALL-E 2 and Stable Diffusion, reflecting stereotypes tied to the script's cultural origin. To mitigate this, the authors propose homoglyph unlearning, a fine-tuning method that makes text encoders robust against such manipulations without full retraining.

ABSTRACT

Models for text-to-image synthesis, such as DALL-E~2 and Stable Diffusion, have recently drawn a lot of interest from academia and the general public. These models are capable of producing high-quality images that depict a variety of concepts and styles when conditioned on textual descriptions. However, these models adopt cultural characteristics associated with specific Unicode scripts from their vast amount of training data, which may not be immediately apparent. We show that by simply inserting single non-Latin characters in a textual description, common models reflect cultural stereotypes and biases in their generated images. We analyze this behavior both qualitatively and quantitatively, and identify a model's text encoder as the root cause of the phenomenon. Additionally, malicious users or service providers may try to intentionally bias the image generation to create racist stereotypes by replacing Latin characters with similarly-looking characters from non-Latin scripts, so-called homoglyphs. To mitigate such unnoticed script attacks, we propose a novel homoglyph unlearning method to fine-tune a text encoder, making it robust against homoglyph manipulations.

Motivation & Objective

• To investigate how non-Latin homoglyphs in text prompts influence cultural bias in text-to-image generation.
• To identify the text encoder as the root cause of script-based cultural bias in multimodal models.
• To demonstrate that single-character homoglyph insertions can subtly but significantly alter image generation toward specific cultural stereotypes.
• To propose a mitigation strategy—homoglyph unlearning—that enhances model robustness against script-based prompt manipulations.
• To raise awareness of hidden biases in generative AI and promote fairness in multimodal systems.

Proposed method

• The authors analyze DALL-E 2 and Stable Diffusion by injecting homoglyphs (e.g., Korean Hangul, Indian Oriya, Arabic) into standard Latin prompts.
• They conduct qualitative and quantitative evaluations to measure shifts in generated image content, focusing on facial features, clothing, and cultural attributes.
• The root cause is traced to the text encoder’s sensitivity to non-Latin script characters, which encode cultural associations from training data.
• A novel fine-tuning method called homoglyph unlearning is proposed, which adapts the text encoder to be invariant to homoglyph substitutions.
• The method is applied without full retraining, preserving model performance while reducing bias from script injection.
• Experiments are conducted using prompts like "a photo of an actress" with homoglyph replacements, and results are analyzed across multiple scripts and models.

Experimental results

Research questions

• RQ1 How do homoglyphs from non-Latin scripts influence the cultural representation in text-to-image generation?
• RQ2 Why does the insertion of a single homoglyph induce significant cultural bias in generated images?
• RQ3 Can the text encoder be made robust to homoglyph manipulations without compromising performance?
• RQ4 To what extent do different Unicode scripts (e.g., Hangul, Oriya, Arabic) affect the cultural stereotypes reflected in generated images?
• RQ5 What are the implications of such biases for fairness, user agency, and potential misuse in generative AI?

Key findings

• Inserting a single homoglyph from non-Latin scripts like Korean Hangul or Indian Oriya into a prompt causes DALL-E 2 and Stable Diffusion to generate images reflecting specific cultural stereotypes, such as East Asian or South Asian facial features and attire.
• The text encoder is identified as the primary source of script-based cultural bias, as it encodes cultural associations from training data.
• Homoglyph unlearning successfully reduces bias induced by homoglyphs, making the text encoder invariant to such character substitutions.
• The method achieves robustness against homoglyph attacks without requiring full retraining, preserving image quality and semantic fidelity.
• The biasing effect is subtle but perceptible and can be exploited to reinforce harmful stereotypes, especially when users are unaware of the manipulation.
• The phenomenon is consistent across multiple models and scripts, indicating a widespread issue in multimodal models trained on diverse internet data.