[Paper Review] Having Beer after Prayer? Measuring Cultural Bias in Large Language Models

Tarek Naous, Michael J. Ryan+2 more|arXiv (Cornell University)|May 23, 2023

Indigenous Studies and Ecology9 citations

TL;DR

The paper introduces CAMeL, a culturally focused benchmark of 20,368 Arab vs. Western entities and 628 prompts to evaluate cultural bias in Arabic LMs, revealing Western bias and cultural unfairness across multilingual and monolingual models, plus insights into pre-training data impacts.

ABSTRACT

As the reach of large language models (LMs) expands globally, their ability to cater to diverse cultural contexts becomes crucial. Despite advancements in multilingual capabilities, models are not designed with appropriate cultural nuances. In this paper, we show that multilingual and Arabic monolingual LMs exhibit bias towards entities associated with Western culture. We introduce CAMeL, a novel resource of 628 naturally-occurring prompts and 20,368 entities spanning eight types that contrast Arab and Western cultures. CAMeL provides a foundation for measuring cultural biases in LMs through both extrinsic and intrinsic evaluations. Using CAMeL, we examine the cross-cultural performance in Arabic of 16 different LMs on tasks such as story generation, NER, and sentiment analysis, where we find concerning cases of stereotyping and cultural unfairness. We further test their text-infilling performance, revealing the incapability of appropriate adaptation to Arab cultural contexts. Finally, we analyze 6 Arabic pre-training corpora and find that commonly used sources such as Wikipedia may not be best suited to build culturally aware LMs, if used as they are without adjustment. We will make CAMeL publicly available at: https://github.com/tareknaous/camel

Motivation & Objective

• Motivate the need for culturally aware LMs in a global context.
• Construct CAMeL to compare Arab vs. Western cultural representations in prompts and entities.
• Evaluate cross-cultural performance of Arabic LMs on generation, NER, sentiment analysis, and text infilling.
• Analyze how Arabic pre-training data sources influence cultural adaptation in LMs.

Proposed method

• Create CAMeL with eight entity types (person names, food dishes, beverages, clothing items, locations, authors, religious places of worship, sports clubs) drawn from Wikidata and CommonCrawl, labeled as Arab vs. Western.
• Generate 628 naturally-occurring Arabic prompts (CAMeL-Co) and 378 neutral prompts (CAMeL-Ag) from Twitter/X contexts.
• Measure cultural bias via stereotype analysis in story generation, cross-cultural fairness in NER and sentiment analysis, and text infilling performance (CBS metric).
• Define Cultural Bias Score (CBS) to compare LM fill probabilities for Western vs. Arab entities across prompts.
• Conduct in-context learning for GPT-type models and fine-tuning for BERT-type models on Arabic NLU benchmarks for NER and sentiment analysis.
• Analyze six Arabic pre-training corpora by training n-gram LMs and computing CBS to assess Western content prevalence.

Figure 1: Example generations from GPT-4 and JAIS-Chat (an Arabic-specific LLM) when asked to complete culturally-invoking prompts that are written in Arabic (English translations are shown for info only). LMs often generate entities that fit in a Western culture (red) instead of the relevant Arab c

Experimental results

Research questions

• RQ1 Do LMs exhibit bias towards Western entities in non-English or non-Western contexts (Arabic prompts)?
• RQ2 Can CAMeL effectively reveal stereotypes, unfairness in NER and sentiment tasks, and adaptations in text infilling across LMs?
• RQ3 How do Arabic pre-training corpora influence cultural adaptation and Western bias in LMs?
• RQ4 Do culturally contextualized prompts (CAMeL-Co) reduce or exacerbate Western bias compared to neutral prompts (CAMeL-Ag)?

Key findings

• CAMeL enables cross-cultural testing across story generation, NER, sentiment analysis, and text infilling.
• LMs show stereotypes in stories, associating Arab names with poverty/traditionalism and Western names with higher status or wealth.
• NER performance is better for Western entities than Arab ones, with larger gaps for locations (up to about 20 F1 points).
• Sentiment analysis shows higher false negatives on sentences with Arab entities, indicating a bias toward associating Arab entities with negative sentiment.
• Culturally contextualized prompts reveal Western bias (CBS 40–60%) across LMs, including monolingual Arabic models, and multilingual LMs show stronger Western bias.
• Prompt-adaptation techniques (especially Arab demonstrations) can reduce CBS, while a culture token has limited effect.
• Arabic pre-training data sources (Wikipedia, international news, web crawls) are Western-centric, correlating with higher CBS; local news and Twitter/X data show lower CBS.

Figure 2: We construct CAMeL , a dataset of masked prompts created from naturally occurring contexts from Twitter/X and comprehensive lists of Arab and Western entities. CAMeL enables various setups for measuring cultural biases in LMs including stereotype assessment, fairness evaluation, and text i