[Paper Review] Analysing Gender Bias in Text-to-Image Models using Object Detection

Harvey Mannering|arXiv (Cornell University)|Jul 16, 2023

Handwritten Text Recognition Techniques4 citations

TL;DR

This paper proposes a novel method to measure gender bias in text-to-image models by generating images with paired gendered prompts (e.g., 'a man holding an item' vs. 'a woman holding an item') and applying object detection to identify gendered object associations. The key finding is that Stable Diffusion exhibits strong gendered associations—men are more likely to be linked with ties, knives, and bicycles, while women are associated with handbags, bottles, and cups—revealed by a chi-squared p-value of 0.000009, indicating significant bias.

ABSTRACT

This work presents a novel strategy to measure bias in text-to-image models. Using paired prompts that specify gender and vaguely reference an object (e.g. "a man/woman holding an item") we can examine whether certain objects are associated with a certain gender. In analysing results from Stable Diffusion, we observed that male prompts generated objects such as ties, knives, trucks, baseball bats, and bicycles more frequently. On the other hand, female prompts were more likely to generate objects such as handbags, umbrellas, bowls, bottles, and cups. We hope that the method outlined here will be a useful tool for examining bias in text-to-image models.

Motivation & Objective

• To develop a new, open-ended method for detecting gender bias in text-to-image models using object detection.
• To investigate how text-to-image models associate specific objects with male or female gendered prompts.
• To compare bias levels between different models, such as Stable Diffusion and DALL·E mini.
• To identify whether training data biases are reflected in model-generated imagery through object frequency analysis.
• To lay the groundwork for detecting other forms of bias (e.g., racial, age-related) using the same framework.

Proposed method

• Using 50 template prompts with gendered subjects (e.g., 'man', 'woman', 'boy', 'girl') and vague object references, 1000 images were generated per model.
• A fixed random seed was used for paired male/female prompts to ensure consistent noise in the diffusion process.
• Object detection was performed using YOLOv3 to identify and count objects in generated images.
• The frequency of detected objects was compared between male and female prompts to assess gendered associations.
• Chi-squared tests were applied to quantify statistical differences between male and female object distribution patterns.
• Results were filtered by minimum occurrence thresholds (9 for Stable Diffusion, 4 for DALL·E mini) to focus on significant findings.

Figure 2: Object detection was run on text-to-image generated images. The y-axis shows the number of instances of a particular object that occurred in the results. Objects are listed on the x-axis. Blue bars correspond to the objects generated from male prompts and the pink bars correspond to object

Experimental results

Research questions

• RQ1 How do text-to-image models associate specific objects with male versus female gendered prompts?
• RQ2 To what extent do models like Stable Diffusion and DALL·E mini exhibit statistically significant gendered object associations?
• RQ3 How does the choice of object detector (e.g., YOLOv3) influence the detection and interpretation of gendered biases?
• RQ4 Can object detection reveal stereotypical gender roles in generated imagery, such as domestic or occupational associations?
• RQ5 How do model-specific training choices (e.g., OpenAI’s bias mitigation in DALL·E) affect observed bias levels?

Key findings

• Stable Diffusion generated ties 38 times from male prompts and only 5 times from female prompts, indicating a strong gendered association.
• Female prompts in Stable Diffusion produced 9 instances of bowls, compared to 3 from male prompts, suggesting a domestic stereotype.
• Bottles and cups were generated 5 and 7 times respectively from female prompts in Stable Diffusion, versus 1 and 2 from male prompts.
• The chi-squared test yielded a p-value of 0.000009 for Stable Diffusion, indicating highly significant gender bias in object distribution.
• DALL·E mini showed a p-value of 0.04172, suggesting less bias than Stable Diffusion, though fewer objects were detected overall.
• Male prompts in both models were more likely to generate objects like knives, trucks, baseball bats, and bicycles, reinforcing traditional gendered roles.