[Paper Review] AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances

D. P. Agarwal, Mor Naaman+1 more|arXiv (Cornell University)|Sep 17, 2024

Computational and Text Analysis Methods11 citations

TL;DR

The study shows Western-centric AI writing suggestions yield greater productivity for Americans than Indians and homogenize Indian writing toward Western styles, reducing cultural nuance.

ABSTRACT

Large language models (LLMs) are being increasingly integrated into everyday products and services, such as coding tools and writing assistants. As these embedded AI applications are deployed globally, there is a growing concern that the AI models underlying these applications prioritize Western values. This paper investigates what happens when a Western-centric AI model provides writing suggestions to users from a different cultural background. We conducted a cross-cultural controlled experiment with 118 participants from India and the United States who completed culturally grounded writing tasks with and without AI suggestions. Our analysis reveals that AI provided greater efficiency gains for Americans compared to Indians. Moreover, AI suggestions led Indian participants to adopt Western writing styles, altering not just what is written but also how it is written. These findings show that Western-centric AI models homogenize writing toward Western norms, diminishing nuances that differentiate cultural expression.

Motivation & Objective

• Assess how Western-centric AI writing suggestions impact users from different cultures (India vs. United States).
• Measure productivity gains and engagement with AI suggestions across cultures.
• Examine whether AI writing suggestions homogenize non-Western writing toward Western styles.
• Discuss cultural harms of embedding Western norms in AI and propose mitigation strategies.

Proposed method

• 2x2 between-subjects experiment with 118 participants (60 Indian, 58 American).
• Conditions: AI suggestions via GPT-4o vs No AI, crossed with participant culture (Indian vs American).
• Four writing tasks designed from Hofstede’s Cultural Onion to elicit explicit and implicit cultural aspects.
• Inline autocomplete suggestions from GPT-4o shown after typing pauses (100 ms); TAB accepts, ESC rejects; logging of interactions.
• Metrics include AI reliance, suggestion acceptance rate, suggestion modification, and writing productivity; NLP measures include Type-Token Ratio and cosine similarity of embeddings.
• Prompts and data collection approved by IRB; demographic survey included to confirm cultural differences (SSVS).

Figure 1. A representation of the potential cultural homogenization from Western-centric AI models

Experimental results

Research questions

• RQ1 RQ1: Does writing with a Western-centric AI provide greater benefits to users from Western cultures compared to those from non-Western cultures?
• RQ2 RQ2: Does writing with a Western-centric AI homogenize the writing styles of non-Western users toward Western styles?

Key findings

• AI boosts productivity for both Indian and American participants, with larger gains for Americans.
• AI causes Indian participants to write more like Americans, indicating Western-style homogenization.
• Indian participants showed higher engagement with AI suggestions (AI reliance) and accepted a larger share of suggestions than Americans.
• A total of 12,015 AI suggestions were shown, with 1,476 accepted (12.3%); 0.6% of rejections were explicit via Escape, while most rejections occurred during flow.
• Top autocomplete suggestions reveal a Western bias, especially for food and festival tasks (Table 4).
• Eight of ten Schwartz values differed significantly between Indian and American participants, supporting cultural distance.