[Paper Review] Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities

Melissa Hall, Laura Gustafson+3 more|arXiv (Cornell University)|Jan 26, 2023

Topic Modeling4 citations

TL;DR

This paper investigates gender-based performance disparities in zero-shot vision-language models (CLIP, Detic, LSeg) across image classification, object detection, and semantic segmentation. It finds that model accuracy and calibration vary significantly by the perceived gender of people in images, with biases mirroring those in pre-trained word embeddings, highlighting risks of bias propagation in open-vocabulary vision systems.

ABSTRACT

We explore the extent to which zero-shot vision-language models exhibit gender bias for different vision tasks. Vision models traditionally required task-specific labels for representing concepts, as well as finetuning; zero-shot models like CLIP instead perform tasks with an open-vocabulary, meaning they do not need a fixed set of labels, by using text embeddings to represent concepts. With these capabilities in mind, we ask: Do vision-language models exhibit gender bias when performing zero-shot image classification, object detection and semantic segmentation? We evaluate different vision-language models with multiple datasets across a set of concepts and find (i) all models evaluated show distinct performance differences based on the perceived gender of the person co-occurring with a given concept in the image and that aggregating analyses over all concepts can mask these concerns; (ii) model calibration (i.e. the relationship between accuracy and confidence) also differs distinctly by perceived gender, even when evaluating on similar representations of concepts; and (iii) these observed disparities align with existing gender biases in word embeddings from language models. These findings suggest that, while language greatly expands the capability of vision tasks, it can also contribute to social biases in zero-shot vision settings. Furthermore, biases can further propagate when foundational models like CLIP are used by other models to enable zero-shot capabilities.

Motivation & Objective

• To investigate whether zero-shot vision-language models exhibit gender-based performance disparities across diverse vision tasks.
• To examine how model calibration—measured by expected calibration error (ECE)—differs by perceived gender of co-occurring persons in images.
• To determine whether observed disparities in vision models align with existing gender biases in pre-trained word embeddings.
• To assess the risk of bias propagation when foundational models like CLIP are used to enable zero-shot capabilities in downstream models.
• To demonstrate that aggregating performance across all concepts can mask significant gender-based disparities, urging disaggregated evaluation.

Proposed method

• Evaluated three zero-shot models: CLIP (ViT-B/32), Detic (for object detection), and LSeg (for semantic segmentation) on multiple datasets including Visual Genome.
• Used text embeddings from language models as classifiers for zero-shot image classification, detection, and segmentation without task-specific fine-tuning.
• Defined perceived gender of persons in images using binarized annotations from datasets, relying on visual perception rather than self-identification.
• Measured performance disparities using average precision (AP) and expected calibration error (ECE) to assess accuracy-confidence alignment across perceived-gender groups.
• Mapped disparities per concept to identify root causes such as differences in object salience (e.g., hair, neckties) and concept co-occurrence patterns across gender groups.
• Compared biases in vision models with gender biases in non-contextual word embeddings (e.g., from word2vec) to assess alignment and propagation.

Experimental results

Research questions

• RQ1 Do zero-shot vision-language models exhibit performance disparities based on the perceived gender of people co-occurring with visual concepts?
• RQ2 How does model calibration—specifically the relationship between confidence and accuracy—differ across perceived-gender groups for the same visual concepts?
• RQ3 To what extent do gender biases in pre-trained word embeddings correlate with biases observed in zero-shot vision-language models?
• RQ4 Does aggregating performance across all concepts obscure significant gender-based disparities in individual concept recognition?
• RQ5 What are the potential root causes of gender-based disparities in zero-shot vision models, particularly in concept-level perception and co-occurrence patterns?

Key findings

• All evaluated models—CLIP, Detic, and LSeg—exhibit distinct performance differences based on the perceived gender of the person co-occurring with a given visual concept, with higher average precision for images of men in many cases.
• Aggregating disparity analyses across all concepts masks significant gender-based performance disparities, underscoring the need for per-concept evaluation.
• Model calibration, measured by expected calibration error (ECE), differs significantly by perceived gender, indicating that models are less confident and accurate for certain gender groups even when representing the same concept.
• Gender-based disparities in vision models strongly align with existing gender biases in non-contextual word embeddings, suggesting that language model biases are transferred to vision tasks.
• Root cause analyses reveal that differences in object salience (e.g., hair, neckties) and concept co-occurrence patterns vary between gender groups, contributing to performance disparities.
• The findings indicate that using language models as supervision in vision tasks can propagate or even amplify social biases, especially when foundational models like CLIP are reused in downstream systems.