[论文解读] Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms

Petter Törnberg, Диляра Валеева+2 more|arXiv (Cornell University)|Oct 5, 2023

Computational and Text Analysis Methods被引用 17

一句话总结

论文将大型语言模型与基于代理的建模相结合，模拟三个社媒平台，具有不同的新闻源算法，测试一种桥接算法，在增加跨党派参与度的同时降低毒性。它使用基于ANES的角色设定和GPT-3.5 提示来在一个模拟日内生成并评估代理互动。

ABSTRACT

Social media is often criticized for amplifying toxic discourse and discouraging constructive conversations. But designing social media platforms to promote better conversations is inherently challenging. This paper asks whether simulating social media through a combination of Large Language Models (LLM) and Agent-Based Modeling can help researchers study how different news feed algorithms shape the quality of online conversations. We create realistic personas using data from the American National Election Study to populate simulated social media platforms. Next, we prompt the agents to read and share news articles - and like or comment upon each other's messages - within three platforms that use different news feed algorithms. In the first platform, users see the most liked and commented posts from users whom they follow. In the second, they see posts from all users - even those outside their own network. The third platform employs a novel "bridging" algorithm that highlights posts that are liked by people with opposing political views. We find this bridging algorithm promotes more constructive, non-toxic, conversation across political divides than the other two models. Though further research is needed to evaluate these findings, we argue that LLMs hold considerable potential to improve simulation research on social media and many other complex social settings.

研究动机与目标

• 研究通过使用LLMs和ABMs来模拟社交媒体是否能够揭示新闻源算法如何影响对话质量。
• 使用ANES数据校准现实的代理人设，以反映美国政治人口统计和媒体消费。
• 测试三种新闻源算法，以评估对跨党派参与度和毒性的影响。
• 提供一个初步评估框架，用于模拟对话并规划未来的人工验证。

提出的方法

• 创建基于LLM的500个代理人设，取自2020 ANES数据，包含政治信念、人口统计和非政治兴趣。
• 基于ANES给代理分配发帖频率和新闻源消费；从2020年7月1日的头条和摘要中为每个代理生成15条故事。
• 运行三个平台仿真，时间线分别为： (1) 仅关注关注者中最受欢迎的帖子，(2) 所有用户的高参与度帖子，(3) 桥接算法优先来自对立方的点赞。
• 允许代理发帖、点赞和评论；根据参与反馈使信息流可见性演变，以模拟实时时间线。
• 使用Perspective API的毒性以及对评论和点赞的跨党派互动指数（E-I）来衡量结果。
• 承认LLM模仿、样本量小、回顾性训练数据的局限性，以及需要人工验证。

Figure 1: Illustration of the model developed in this paper, which combines Large Language Models and Agent-Based Models to simulate the impact of bridging algorithms on social media discourse. Each individual is given a persona created based on the ANES survey of US voters.

实验结果

研究问题

• RQ1 LLM为基础的ABMs是否能够足以模拟社媒，从而比较替代新闻源算法？
• RQ2 桥接算法是否在不提高毒性的情况下增加跨党派参与，相较于传统信息流设计？
• RQ3 以参与驱动的时间线在模拟的美国政治背景下如何影响毒性和跨党派对话？

主要发现

• 平台1 由于回音室效应，毒性低且跨党派互动最小。
• 平台2 显示更高的跨党派互动和毒性，接近所研究时期的Twitter级别毒性。
• 平台3（桥接）展现最有利的结果：跨党派互动更多，三个平台中毒性最低。
• 桥接时间线强调跨切话题，相较于其他模型减少滥用语言。
• 定性示例表明桥接可以促进跨党派话语，主题包括媒体中的代表性等。

Figure 2: Excerpt of the generated timeline from Platform 1.