[论文解读] Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments

Carlos Carrasco-Farré|arXiv (Cornell University)|Apr 14, 2024

European and International Law Studies被引用 12

一句话总结

本文比较了 LLMs 生成的论点与人工生成的论点，显示 LLMs 需要更高的认知负荷，并使用更多的道德语言，但情感内容方面没有差异。

ABSTRACT

Large Language Models (LLMs) are already as persuasive as humans. However, we know very little about how they do it. This paper investigates the persuasion strategies of LLMs, comparing them with human-generated arguments. Using a dataset of 1,251 participants in an experiment, we analyze the persuasion strategies of LLM-generated and human-generated arguments using measures of cognitive effort (lexical and grammatical complexity) and moral-emotional language (sentiment and moral analysis). The study reveals that LLMs produce arguments that require higher cognitive effort, exhibiting more complex grammatical and lexical structures than human counterparts. Additionally, LLMs demonstrate a significant propensity to engage more deeply with moral language, utilizing both positive and negative moral foundations more frequently than humans. In contrast with previous research, no significant difference was found in the emotional content produced by LLMs and humans. These findings contribute to the discourse on AI and persuasion, highlighting the dual potential of LLMs to both enhance and undermine informational integrity through communication strategies for digital persuasion.

研究动机与目标

• 研究 LLMs 相对于人类的说服方式。
• 通过词汇和语法复杂性分析论证中的认知努力。
• 考察 LLM 与人类论证中道德-情感语言的使用。

提出的方法

• 在实验中使用包含 1,251 名参与者的数据集。
• 通过词汇/语法复杂性分析 LLM 生成和人类生成的论点的认知努力。
• 用道德基础分析和情感分析评估道德语言。
• 比较 LLM 与人类论证中的情感内容。

实验结果

研究问题

• RQ1 在认知努力指标上，LLM 生成的论点是否与人类论点不同？
• RQ2 LLM 论点是否比人类论点使用更多的道德语言？
• RQ3 LLM 与人类论点在情感内容上是否存在差异？

主要发现

• LLMs 生成的论点通过更复杂的语法和词汇，需要更高的认知努力。
• LLMs 表现出显著倾向于更深入地涉及道德语言，比人类更频繁地使用正向和负向的道德基础。
• LLM 与人类论点在情感内容上没有显著差异。