[Paper Review] Bias of AI-Generated Content: An Examination of News Produced by Large Language Models

Fang Xiao, Shangkun Che+4 more|arXiv (Cornell University)|Sep 18, 2023

Computational and Text Analysis Methods16 citations

TL;DR

The paper evaluates gender and racial biases in AI-generated news from seven LLMs by comparing AIGC to NYT/Reuters articles at word, sentence, and document levels, including bias under biased prompts and RLHF effects.

ABSTRACT

Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.

Motivation & Objective

• Proxy unbiased content with high-quality news articles from The New York Times and Reuters as reference content.
• Generate AIGC using headlines as prompts and compare word-level, sentence-level, and document-level biases to reference content.
• Analyze bias under biased prompts and assess model resistance to biased prompts.
• Evaluate how model size and RLHF influence bias across gender and racial groups.

Proposed method

• Collect 8,629 NYT and Reuters news articles from Dec 2022 to Apr 2023 as reference content.
• Apply each LLM to generate news content using the article headlines as prompts.
• Measure word-level bias using Wasserstein distance between population-group word distributions in AIGC vs reference content.
• Assess sentence-level bias via sentiments and toxicities for gender/race-related sentences.
• Assess document-level bias via semantics and topics for gender/race-related content.
• Examine bias under biased prompts by injecting gender-biased messages into prompts and evaluate models' resistance to biased prompts.

Experimental results

Research questions

• RQ1 How does AIGC from representative LLMs differ from high-quality reference news in gender- and race-related word usage?
• RQ2 What are the sentence- and document-level biases in AIGC regarding gender and race, including sentiment and toxicity?
• RQ3 How does AIGC respond to biased prompts, and to what extent do models resist or propagate such bias?
• RQ4 Does model size or RLHF (as in ChatGPT) mitigate bias across word, sentence, and document levels?

Key findings

• All evaluated LLMs produce AIGC with substantial gender and racial bias relative to NYT/Reuters references at word, sentence, and document levels.
• ChatGPT generally shows the lowest bias among the tested models, aided by reinforcement learning from human feedback (RLHF).
• RLHF contributes to reduced word- and document-level bias and enables content refusal under biased prompts for ChatGPT, though biased prompts can still yield highly biased outputs when not filtered.
• Black prejudice is particularly pronounced across models at the word level, with significant decreases in Black-race word usage in AIGC compared to references.
• Bias tends to diminish as model size increases among GPT-family models, with RLHF further aiding bias reduction across metrics.
• Document-level analyses show notable gender bias and racial bias across models, with ChatGPT often performing best but not barrier-proof against biased prompts.