[Paper Review] OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMs

Patrick Haller, Ansar Aynetdinov+1 more|arXiv (Cornell University)|Sep 7, 2023

Text Readability and Simplification4 citations

TL;DR

OpinionGPT introduces a web-based demonstration that explicitly models and compares multiple societal biases—such as political, geographic, gender, and age-based perspectives—in instruction-tuned LLMs. By fine-tuning a LLaMA model on Reddit-based instruction-response pairs labeled by demographic biases, the system generates side-by-side responses reflecting distinct viewpoints, offering transparency over bias rather than suppression.

ABSTRACT

Instruction-tuned Large Language Models (LLMs) have recently showcased remarkable ability to generate fitting responses to natural language instructions. However, an open research question concerns the inherent biases of trained models and their responses. For instance, if the data used to tune an LLM is dominantly written by persons with a specific political bias, we might expect generated answers to share this bias. Current research work seeks to de-bias such models, or suppress potentially biased answers. With this demonstration, we take a different view on biases in instruction-tuning: Rather than aiming to suppress them, we aim to make them explicit and transparent. To this end, we present OpinionGPT, a web demo in which users can ask questions and select all biases they wish to investigate. The demo will answer this question using a model fine-tuned on text representing each of the selected biases, allowing side-by-side comparison. To train the underlying model, we identified 11 different biases (political, geographic, gender, age) and derived an instruction-tuning corpus in which each answer was written by members of one of these demographics. This paper presents OpinionGPT, illustrates how we trained the bias-aware model and showcases the web application (available at https://opiniongpt.informatik.hu-berlin.de).

Motivation & Objective

• To investigate how inherent biases in training data shape LLM responses, particularly in politically and demographically sensitive domains.
• To challenge the dominant paradigm of bias suppression by instead making biases explicit and transparent in model outputs.
• To develop a tool that enables users to compare how the same question is answered from multiple biased perspectives.
• To provide researchers and the public with a platform to study subjectivity, language variation, and bias in NLP systems.
• To raise awareness about the role of demographic and cultural perspectives in language generation without endorsing any specific viewpoint.

Proposed method

• Constructed a bias-aware instruction-tuning corpus by collecting Reddit posts and responses from subreddits aligned with specific demographic groups (e.g., 'r/conservative', 'r/DebatePolitics', 'r/AskGermany').
• Annotated each response with its associated bias category, including political (liberal, conservative), geographic (USA, Germany, Middle East, Latin America), age (teenager, over 30, over 45), and gender (male, female).
• Fine-tuned a LLaMA model using instruction-tuning on the curated dataset, where each response was associated with a specific bias label.
• Designed a web interface allowing users to input a question and select multiple bias categories, triggering the model to generate distinct responses per selected bias.
• Implemented a prompt engineering strategy to condition the model on the selected bias during inference, ensuring each response reflects the corresponding demographic perspective.
• Employed anonymized and de-identified user interaction data, with no model release to prevent misuse, and restricted access via a protected API for research use only.

Experimental results

Research questions

• RQ1 How do different demographic biases—political, geographic, age, and gender—affect the responses generated by instruction-tuned LLMs?
• RQ2 To what extent can a single LLM be fine-tuned to generate contextually appropriate and distinct responses for multiple predefined bias categories?
• RQ3 Can making biases explicit in model outputs improve transparency and understanding of subjectivity in NLP systems?
• RQ4 What are the risks of bias leakage and conflation during fine-tuning when training on data from online forums like Reddit?
• RQ5 How can bias-aware models be ethically deployed to foster discourse without promoting harmful ideologies?

Key findings

• The OpinionGPT model successfully generates distinct, contextually relevant responses for each selected bias category, demonstrating that fine-tuning on bias-labeled data enables controlled perspective generation.
• Responses varied significantly across biases—for example, the question 'Give two examples of reputable TV news channels' elicited 'Fox News' from conservative, 'the Verge' from liberal, 'CNN' from American, and 'Tagesschau' from German perspectives.
• The model exhibits nuanced responses that reflect the linguistic and cultural characteristics associated with each bias group, indicating effective capture of perspective-specific language patterns.
• Despite success, the model inherits a global layer of bias from Reddit’s user base, meaning 'Americans' or 'Germans' in the model represent only those who post on specific subreddits, not the general population.
• Instances of bias leakage were observed, where responses from one bias group occasionally reflected characteristics of another, suggesting potential confounding during fine-tuning.
• The project highlights the ethical risks of deploying bias-aware models, leading to the decision not to release the model publicly, but to provide access via a protected API to researchers only.