[Paper Review] ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages

Sourojit Ghosh, Aylin Caliskan|arXiv (Cornell University)|May 17, 2023

Artificial Intelligence in Healthcare and EducationMedicine12 citations

TL;DR

The paper reveals that ChatGPT exhibits gender bias in translation between English and Bengali and five other low-resource languages, often defaulting to gendered pronouns and ignoring non-gendered pronouns.

ABSTRACT

In this multicultural age, language translation is one of the most performed tasks, and it is becoming increasingly AI-moderated and automated. As a novel AI system, ChatGPT claims to be proficient in such translation tasks and in this paper, we put that claim to the test. Specifically, we examine ChatGPT's accuracy in translating between English and languages that exclusively use gender-neutral pronouns. We center this study around Bengali, the 7$^{th}$ most spoken language globally, but also generalize our findings across five other languages: Farsi, Malay, Tagalog, Thai, and Turkish. We find that ChatGPT perpetuates gender defaults and stereotypes assigned to certain occupations (e.g. man = doctor, woman = nurse) or actions (e.g. woman = cook, man = go to work), as it converts gender-neutral pronouns in languages to `he' or `she'. We also observe ChatGPT completely failing to translate the English gender-neutral pronoun `they' into equivalent gender-neutral pronouns in other languages, as it produces translations that are incoherent and incorrect. While it does respect and provide appropriately gender-marked versions of Bengali words when prompted with gender information in English, ChatGPT appears to confer a higher respect to men than to women in the same occupation. We conclude that ChatGPT exhibits the same gender biases which have been demonstrated for tools like Google Translate or MS Translator, as we provide recommendations for a human centered approach for future designers of AIs that perform language translation to better accommodate such low-resource languages.

Motivation & Objective

• Assess ChatGPT's translation accuracy between English and Bengali and five other low-resource languages with gender-neutral pronouns.
• Evaluate whether ChatGPT enforces gender defaults and stereotypes in translations of occupations and actions.
• Examine how ChatGPT handles gender-neutral pronouns such as they across targeted languages.
• Provide recommendations for human-centered design to better accommodate low-resource languages in AI translators.

Proposed method

• Analyze translations from English to Bengali and five additional low-resource languages (Farsi, Malay, Tagalog, Thai, Turkish) using ChatGPT.
• Identify instances where gender-neutral pronouns are translated to gendered forms (he/she) or are incoherently translated.
• Compare gender-marked translations when gender information is provided in English vs. non-gendered contexts.
• Assess relative treatment of men vs. women in same occupations within translations.
• Synthesize findings to propose design recommendations for multilingual AI translation tools.

Experimental results

Research questions

• RQ1 Does ChatGPT translate gender-neutral pronouns into genderedpronouns in Bengali and the other studied languages?
• RQ2 Does the model propagate gender stereotypes in occupation- or action-related translations?
• RQ3 How does ChatGPT treat gender-neutral pronouns like they across Bengali and the five low-resource languages?
• RQ4 What design recommendations can minimize gender bias in AI translation systems for low-resource languages?

Key findings

• ChatGPT often converts gender-neutral pronouns into he or she in translations for Bengali and the other languages studied.
• The model propagates gender-default stereotypes (e.g., certain occupations linked to specific genders) in translations.
• ChatGPT fails to translate English gender-neutral pronouns like they into equivalent gender-neutral forms in other languages, yielding incoherent or incorrect translations.
• When prompted with English gender information, ChatGPT provides gender-marked Bengali equivalents, but appears to respect men more than women in the same occupation.
• Overall, the biases observed align with prior findings for other translation tools, underscoring the need for human-centered design in AI translation.