[Paper Review] Large Language Models are Geographically Biased

Rohin Manvi, Samar Khanna+3 more|arXiv (Cornell University)|Feb 5, 2024

Computational and Text Analysis Methods13 citations

TL;DR

The paper shows that LLMs can make accurate zero-shot geospatial predictions but exhibit geographic biases, especially against low-socioeconomic regions on sensitive subjective topics; it introduces a bias metric and analyzes variance across models.

ABSTRACT

Large Language Models (LLMs) inherently carry the biases contained in their training corpora, which can lead to the perpetuation of societal harm. As the impact of these foundation models grows, understanding and evaluating their biases becomes crucial to achieving fairness and accuracy. We propose to study what LLMs know about the world we live in through the lens of geography. This approach is particularly powerful as there is ground truth for the numerous aspects of human life that are meaningfully projected onto geographic space such as culture, race, language, politics, and religion. We show various problematic geographic biases, which we define as systemic errors in geospatial predictions. Initially, we demonstrate that LLMs are capable of making accurate zero-shot geospatial predictions in the form of ratings that show strong monotonic correlation with ground truth (Spearman's $ρ$ of up to 0.89). We then show that LLMs exhibit common biases across a range of objective and subjective topics. In particular, LLMs are clearly biased against locations with lower socioeconomic conditions (e.g. most of Africa) on a variety of sensitive subjective topics such as attractiveness, morality, and intelligence (Spearman's $ρ$ of up to 0.70). Finally, we introduce a bias score to quantify this and find that there is significant variation in the magnitude of bias across existing LLMs. Code is available on the project website: https://rohinmanvi.github.io/GeoLLM

Motivation & Objective

• Demonstrate zero-shot geospatial prediction capabilities of LLMs using coordinates as ground-truth-aligned prompts.
• Show that LLMs exhibit geographic biases across objective and sensitive subjective topics.
• Quantify bias magnitude with a metric that combines rank correlation, rating dispersion, and response rate across models.
• Compare bias levels across multiple popular LLMs (e.g., GPT-4 Turbo, GPT-3.5 Turbo, Gemini Pro, Mixtral, Llama 2).

Proposed method

• Prompt-based zero-shot elicitation of location-wise ratings for various topics using a prefix plus a GeoLLM-style prompt.
• Use Spearman’s ρ to measure monotonic alignment between LLM ratings and ground-truth geospatial data.
• Visualize predictions on global maps using rank-based analyses and rank errors to reveal systematic biases.
• Define a bias score B_y(x) that multiplies Spearman’s ρ with MAD of ratings and the model’s answer rate to quantify bias on sensitive subjective topics.
• Anchor bias measurements with a distribution such as Infant Mortality to relate model ratings to socioeconomic proxies.
• Evaluate the added value of using the expected value of ratings (logprobs) versus the most probable rating for zero-shot predictions.

Figure 1: The mean rank plots illustrate agreement across LLM predictions, with areas of green and red highlighting regions consistently rated higher or lower respectively. For objective topics, the maps demonstrate the zero-shot geographic knowledge of LLMs. The sensitive subjective topics reveal a

Experimental results

Research questions

• RQ1 Can LLMs perform accurate zero-shot geospatial predictions across a range of objective topics?
• RQ2 Do LLMs exhibit geographic biases across objective and sensitive subjective topics, and how do these biases vary across models?
• RQ3 How can we quantify geographic bias in LLM outputs for sensitive topics, and what factors influence its magnitude?
• RQ4 Do different LLMs show varying levels of geographic bias, and can bias be reduced by using logprob-based expectations?
• RQ5 What is the relationship between bias in ratings and proxies for socioeconomic conditions (e.g., infant mortality)?

Key findings

• LLMs achieve strong monotonic correlation with ground-truth geospatial data in zero-shot predictions, with Spearman’s ρ up to 0.89 for some topics.
• LLMs display consistent geographic biases across objective topics, such as Africa and India showing underestimation of population density and Southeast Asia showing underestimation of infant mortality/risk proxies.
• On sensitive subjective topics (e.g., attractiveness, morality, intelligence), LLMs show bias against areas with lower socioeconomic conditions, with correlations to infant survival rate reaching up to 0.70.
• There is substantial variation in bias magnitude across models; GPT-4 Turbo and Llama 2 70B appear relatively less biased compared to some others (e.g., Gemini Pro).
• Using the expected value of ratings with logprobs improves prediction performance and can reveal subtler biases not captured by the most likely rating.
• The proposed bias score B_y(x) combines rank correlation, rating dispersion (MAD), and answer rate to quantify geographic bias on sensitive topics.

Figure 3: Zero-shot GPT-4 Turbo comparison with ground truth.