[Paper Review] She Elicits Requirements and He Tests: Software Engineering Gender Bias in Large Language Models

Christoph Treude, Hideaki Hata|arXiv (Cornell University)|Mar 17, 2023

Hate Speech and Cyberbullying Detection4 citations

TL;DR

This study uses back-translation through Finnish to detect implicit gender bias in large language models related to 56 software engineering tasks. It reveals that testing is almost exclusively associated with 'he' (100%), while requirements elicitation is linked with 'he' in only 6% of cases, exposing a pronounced gendered association in task roles that reflects and reinforces societal stereotypes.

ABSTRACT

Implicit gender bias in software development is a well-documented issue, such as the association of technical roles with men. To address this bias, it is important to understand it in more detail. This study uses data mining techniques to investigate the extent to which 56 tasks related to software development, such as assigning GitHub issues and testing, are affected by implicit gender bias embedded in large language models. We systematically translated each task from English into a genderless language and back, and investigated the pronouns associated with each task. Based on translating each task 100 times in different permutations, we identify a significant disparity in the gendered pronoun associations with different tasks. Specifically, requirements elicitation was associated with the pronoun "he" in only 6% of cases, while testing was associated with "he" in 100% of cases. Additionally, tasks related to helping others had a 91% association with "he" while the same association for tasks related to asking coworkers was only 52%. These findings reveal a clear pattern of gender bias related to software development tasks and have important implications for addressing this issue both in the training of large language models and in broader society.

Motivation & Objective

• To investigate how implicit gender bias in large language models affects associations with specific software engineering tasks.
• To identify which development tasks are disproportionately linked to male pronouns like 'he' through data mining techniques.
• To understand how gendered language patterns in LLMs may reinforce stereotypes in software engineering roles.
• To provide evidence for targeted interventions in model training and task assignment to reduce gender bias.
• To demonstrate the utility of back-translation with genderless languages as a method for detecting subtle, implicit gender bias in NLP systems.

Proposed method

• Back-translation: Each software development task is first translated from English into Finnish (a genderless language) and then back to English using the DeepL translator.
• The process is repeated 100 times with randomized permutations of the task list to minimize context-dependent bias in translation.
• Pronoun associations in the final English sentences are analyzed to determine the frequency of 'he', 'she', 'he or she', 'he/she', and other forms.
• The study uses the DeepL API for translation, leveraging its high-quality output to detect implicit gendered associations in model-generated text.
• The method relies on the assumption that if a task is consistently translated with 'he' after back-translation, it reflects a latent gender bias in the model’s training data.
• Results are aggregated across 100 runs to ensure statistical robustness and reduce noise from individual translation variations.

Experimental results

Research questions

• RQ1 To what extent are different software engineering tasks associated with male pronouns like 'he' in large language models?
• RQ2 How does the gendered pronoun association vary across tasks such as requirements elicitation, testing, and mentoring?
• RQ3 Are tasks involving communication or collaboration more likely to be associated with 'she' or 'he'?
• RQ4 Does the back-translation method effectively reveal implicit gender bias in LLMs that is not apparent in raw text?
• RQ5 What role does task context—such as customer interaction or technical autonomy—play in shaping pronoun associations?

Key findings

• Testing is associated with the pronoun 'he' in 100% of back-translations, indicating a strong and consistent gender bias toward men in this role.
• Requirements elicitation is associated with 'he' in only 6% of cases, suggesting a weaker or more neutral association with male pronouns.
• Tasks involving helping others are associated with 'he' in 91% of cases, indicating a pronounced bias toward men in supportive or collaborative roles.
• Tasks related to asking coworkers are associated with 'he' in only 52% of cases, showing a significantly weaker link to male pronouns.
• Commenting and learning are exclusively associated with 'he' in all 100 back-translations, highlighting extreme bias in these cognitive or knowledge-sharing tasks.
• The pronoun 'she' is more frequently associated with customer-facing or communication-intensive tasks, such as mentoring and meetings, suggesting a gendered role assignment in interpersonal software engineering activities.