[论文解读] AI Chat Assistants can Improve Conversations about Divisive Topics

Lisa P. Argyle, Ethan C. Busby+5 more|arXiv (Cornell University)|Feb 14, 2023

Social Media and Politics被引用 12

一句话总结

基于GPT-3的聊天助手为关于枪支政策的在线对话提供实时改写建议，提高被认为的理解程度和民主互惠，特别是对话伴侣，同时不改变政策态度。

ABSTRACT

A rapidly increasing amount of human conversation occurs online. But divisiveness and conflict can fester in text-based interactions on social media platforms, in messaging apps, and on other digital forums. Such toxicity increases polarization and, importantly, corrodes the capacity of diverse societies to develop efficient solutions to complex social problems that impact everyone. Scholars and civil society groups promote interventions that can make interpersonal conversations less divisive or more productive in offline settings, but scaling these efforts to the amount of discourse that occurs online is extremely challenging. We present results of a large-scale experiment that demonstrates how online conversations about divisive topics can be improved with artificial intelligence tools. Specifically, we employ a large language model to make real-time, evidence-based recommendations intended to improve participants' perception of feeling understood in conversations. We find that these interventions improve the reported quality of the conversation, reduce political divisiveness, and improve the tone, without systematically changing the content of the conversation or moving people's policy attitudes. These findings have important implications for future research on social media, political deliberation, and the growing community of scholars interested in the place of artificial intelligence within computational social science.

研究动机与目标

• 证明基于AI的实时改写是否能够提升在线政治对话中的感知理解。
• 测试此类干预是否在不改变参与者政策态度的情况下促进对对手的民主互惠。
• 评估AI引导是否影响对话语气而不改变话题内容。
• 评估当AI干预应用于一名参与者与其伙伴时的差异效应。
• 探索AI辅助的对话干预在存在分歧主题上的可扩展性。

提出的方法

• 在美国就枪支政策辩论开展大规模在线现场实验。
• 在实时对话中使用基于GPT-3的聊天助手提出三种情境感知的改写（重述、确认、礼貌）。
• 在每个对话中随机指派一名参与者接收AI建议，而另一方可能受到也可能不受到干预。
• 通过对话后问卷测量对话质量和民主互惠。
• 分析文本以验证语气变化和话题保持，并对治疗效应使用安慰剂对照的子组分析。

Figure 1: Treated Conversation Flow : Respondents write messages unimpeded until one partner receives a rephrasing prompt for the first message longer than four words, and every other conversational turn thereafter. The chat assistant intercepts the treated user’s message, using GPT-3 to propose evi

实验结果

研究问题

• RQ1 AI辅助改写是否提高参与者在与政治对手的对话中被理解的感知？
• RQ2 AI生成的干预是否增强对对手在政治体系中的民主互惠？
• RQ3 这些干预是否改变参与者或其伙伴的实质性政策态度？
• RQ4 相对于受干预的个体，干预对象的伙伴是否表现出更大的效应？

主要发现

• 参与者选择的AI改写在信息中增加了礼貌、确认和重述等特征。
• 受干预个体的伙伴报告了更高的对话质量和民主互惠。
• 完整干预使伙伴的民主互惠提升约6个百分点。
• 没有证据表明AI干预改变了参与者的枪支政策态度。
• 在初始分歧较高的对话中效应最强，且仅在短期内持续。

Figure 2: Text analysis of rephrased messages tone : Marginal difference, with 95% confidence intervals, between rephrased message scores on five politeness package features, and baseline scores from participants’ original messages they would have sent had they not chosen the rephrasing.