[Paper Review] Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recency

Yashar Deldjoo|arXiv (Cornell University)|Jan 19, 2024

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper investigates biases in ChatGPT-based recommender systems, demonstrating that prompt design—particularly role-based prompts—significantly enhances fairness, diversity, and recency in movie recommendations. It finds that GPT-based models outperform traditional collaborative filtering in recommending newer, genre-diverse films (especially post-2000 dramas and comedies), with inclusion of user ratings boosting accuracy and stability.

ABSTRACT

This paper explores the biases in ChatGPT-based recommender systems, focusing on provider fairness (item-side fairness). Through extensive experiments and over a thousand API calls, we investigate the impact of prompt design strategies-including structure, system role, and intent-on evaluation metrics such as provider fairness, catalog coverage, temporal stability, and recency. The first experiment examines these strategies in classical top-K recommendations, while the second evaluates sequential in-context learning (ICL). In the first experiment, we assess seven distinct prompt scenarios on top-K recommendation accuracy and fairness. Accuracy-oriented prompts, like Simple and Chain-of-Thought (COT), outperform diversification prompts, which, despite enhancing temporal freshness, reduce accuracy by up to 50%. Embedding fairness into system roles, such as "act as a fair recommender," proved more effective than fairness directives within prompts. Diversification prompts led to recommending newer movies, offering broader genre distribution compared to traditional collaborative filtering (CF) models. The second experiment explores sequential ICL, comparing zero-shot and few-shot ICL. Results indicate that including user demographic information in prompts affects model biases and stereotypes. However, ICL did not consistently improve item fairness and catalog coverage over zero-shot learning. Zero-shot learning achieved higher NDCG and coverage, while ICL-2 showed slight improvements in hit rate (HR) when age-group context was included. Our study provides insights into biases of RecLLMs, particularly in provider fairness and catalog coverage. By examining prompt design, learning strategies, and system roles, we highlight the potential and challenges of integrating LLMs into recommendation systems. Further details can be found at https://github.com/yasdel/Benchmark_RecLLM_Fairness.

Motivation & Objective

• To investigate how prompt design strategies influence recommendation quality in ChatGPT-based recommender systems.
• To compare the performance of GPT-based models against traditional collaborative filtering (CF) baselines in terms of accuracy, fairness, diversity, and temporal stability.
• To analyze the impact of including user ratings in prompts on recommendation personalization and reliability.
• To understand why GPT-based models favor newer and more diverse genres compared to CF models.
• To identify opportunities for improving RecLLM performance through constrained prompting, few-shot learning, and hybrid RAG frameworks.

Proposed method

• Employed a variety of prompt design strategies, including role-based prompts (e.g., 'you are a fair recommender') and reasoning paradigms (e.g., Chain-of-Thought).
• Evaluated GPT-based models under implicit (no ratings) and explicit (with ratings) prompting scenarios to assess personalization impact.
• Compared recommendation outcomes across multiple metrics: NDCG, Recall, provider fairness, genre diversity, and temporal freshness (post-2000 film preference).
• Used confidence intervals to assess statistical stability of performance across runs.
• Constrained GPT outputs to only select from a predefined test set of items to evaluate accuracy and relevance under dataset boundaries.
• Proposed future integration of GPT with CF via a Retrieval-Augmented Generation (RAG) framework to balance personalization and exploration.

Figure 1 . Conceptual idea behind our work

Experimental results

Research questions

• RQ1 How do different prompt design strategies affect fairness, diversity, and accuracy in GPT-based movie recommender systems?
• RQ2 To what extent do GPT-based models outperform traditional collaborative filtering models in recommending newer and more diverse films?
• RQ3 What is the impact of including explicit user ratings in prompts on the stability and accuracy of GPT-based recommendations?
• RQ4 Why do GPT-based models show a preference for post-2000 films and genres like Drama and Comedy over Action and Adventure?
• RQ5 How can prompt constraints and few-shot learning improve the relevance and accuracy of GPT-based recommendations?

Key findings

• Role-based prompts significantly improved provider fairness and diversity, effectively mitigating popularity bias in recommendations.
• The 'Include-rating' GPT model achieved the highest performance with an NDCG of 0.013230 and Recall of 0.014721, outperforming other GPT-based scenarios.
• GPT-based models demonstrated a strong preference for newer films, particularly those released after 2000, indicating temporal freshness in recommendations.
• GPT models showed a genre bias toward 'Drama', 'Comedy', and 'Romance'—contrasting with CF models’ preference for 'Action' and 'Adventure'—likely due to training data exposure to recent cultural trends.
• Chain-of-Thought (COT) and simple prompting strategies yielded the highest accuracy among all tested GPT-based approaches.
• The confidence intervals for NDCG (0.009737–0.017012) and Recall (0.010261–0.019795) in the 'Include-rating' model confirmed its robust and reliable performance.

(a) NDCG@10.