[论文解读] Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System

Yunfan Gao, Tao Sheng+4 more|arXiv (Cornell University)|Mar 25, 2023

Topic Modeling被引用 64

一句话总结

Chat-Rec 通过上下文提示在推荐系统中提升了对多轮、互动式、可解释性推荐的能力，包括跨域和冷启动处理。

ABSTRACT

Large language models (LLMs) have demonstrated their significant potential to be applied for addressing various application tasks. However, traditional recommender systems continue to face great challenges such as poor interactivity and explainability, which actually also hinder their broad deployment in real-world systems. To address these limitations, this paper proposes a novel paradigm called Chat-Rec (ChatGPT Augmented Recommender System) that innovatively augments LLMs for building conversational recommender systems by converting user profiles and historical interactions into prompts. Chat-Rec is demonstrated to be effective in learning user preferences and establishing connections between users and products through in-context learning, which also makes the recommendation process more interactive and explainable. What's more, within the Chat-Rec framework, user's preferences can transfer to different products for cross-domain recommendations, and prompt-based injection of information into LLMs can also handle the cold-start scenarios with new items. In our experiments, Chat-Rec effectively improve the results of top-k recommendations and performs better in zero-shot rating prediction task. Chat-Rec offers a novel approach to improving recommender systems and presents new practical scenarios for the implementation of AIGC (AI generated content) in recommender system studies.

研究动机与目标

• 解决传统推荐系统在交互性和可解释性方面的不足
• 提出一种基于提示的界面，利用大语言模型从上下文中学习用户偏好
• 通过在不同物品领域之间迁移用户偏好，实现跨域推荐
• 通过整合外部信息解决新物品的冷启动挑战
• 在真实数据上展示对 top-k 推荐和零-shot 评分预测的有效性

提出的方法

• 通过将用户画像和历史转换为引导 ChatGPT 进行推荐的 prompts，连接传统推荐与 LLMs
• 使用一个以输入为驱动的提示构造器，汇总用户历史、画像、查询和对话历史以生成定制化 prompts
• 通过上下文学习与推理让 LLM 过滤并重新排序候选集
• 通过引入外部项目信息来生成新物品的嵌入并检索相似项，从而处理冷启动
• 通过使 LLM 根据电影偏好来推荐非电影项来实现跨域迁移
• 在 MovieLens 100K 上结合多种 GPT-3.5 家族模型评估 top-k 推荐和零-shot 评分预测

Figure 1 : Overview of Chat-Rec . The left side shows a dialogue between a user and ChatGPT. The middle side shows the flowchart to how Chat-Rec links traditional recommender systems with conversational AI such as ChatGPT. The right side describes the specific judgment in the process.

实验结果

研究问题

• RQ1 在不进行额外训练的情况下，增加上下文学习的 LLM 是否能提升 top-k 推荐性能？
• RQ2 提示设计如何影响交互式、可解释性推荐的质量？
• RQ3 系统能否有效地支持跨域和冷启动的推荐？
• RQ4 候选集及其排序在最终由 LLM 产生的排序中扮演怎样的角色？
• RQ5 不同的 LLM 主干（GPT-3.5 变体）在推荐和评分预测任务上是否存在显著差异？

主要发现

模型	Precision	Recall	NDCG
LightFM	0.2830	0.1410	0.2846
LightGCN	0.3030	0.1455	0.3425
Chat-Rec (gpt-3.5-turbo)	0.3103	0.1279	0.3696
Chat-Rec (text-davinci-003)	0.3240 (+6.93%)	0.1404 (-3.51%)	0.3802 (+11.01%)
Chat-Rec (text-davinci-002)	0.3031	0.1240	0.3629

• Chat-Rec 在 MovieLens 100K 上对比 LightGCN，在测试的 GPT-3.5 模型中提升了 top-k 推荐指标。
• text-davinci-003 在 top-k 结果中表现最佳，精度为 0.3240，召回率为 0.1404，NDCG 为 0.3802。
• 在评分预测方面，text-davinci-003 在测试模型中取得最佳 RMSE 0.785 和 MAE 0.593。
• 消融分析表明提示设计与包含 top-1 基线背景对 NDCG 有显著影响，温度和提示顺序也会影响性能。
• Chat-Rec 能显著重新排序和精炼一个大型候选集，在不进行显式推荐器训练的情况下提升相关性。

Figure 2 : Case study of interactive recommendation. It shows two conversations between different users and LLM . Where the user profile and historical users are converted into corresponding prompts for personalized recommendations, but the input of this part of the prompts is not visible to the use