[论文解读] Labeling Messages as AI-Generated Does Not Reduce Their Persuasive Effects

Isabel O. Gallegos, Chen Shani+5 more|ArXiv.org|Apr 14, 2025

Psychology of Moral and Emotional Judgment被引用 3

一句话总结

研究发现将AI生成的消息标注为AI、人工撰写或未标注，对其在四个政策领域的说服力没有显著影响。

ABSTRACT

As generative artificial intelligence (AI) enables the creation and dissemination of information at massive scale and speed, it is increasingly important to understand how people perceive AI-generated content. One prominent policy proposal requires explicitly labeling AI-generated content to increase transparency and encourage critical thinking about the information, but prior research has not yet tested the effects of such labels. To address this gap, we conducted a survey experiment (N=1601) on a diverse sample of Americans, presenting participants with an AI-generated message about several public policies (e.g., allowing colleges to pay student-athletes), randomly assigning whether participants were told the message was generated by (a) an expert AI model, (b) a human policy expert, or (c) no label. We found that messages were generally persuasive, influencing participants' views of the policies by 9.74 percentage points on average. However, while 94.6% of participants assigned to the AI and human label conditions believed the authorship labels, labels had no significant effects on participants' attitude change toward the policies, judgments of message accuracy, nor intentions to share the message with others. These patterns were robust across a variety of participant characteristics, including prior knowledge of the policy, prior experience with AI, political party, education level, or age. Taken together, these results imply that, while authorship labels would likely enhance transparency, they are unlikely to substantially affect the persuasiveness of the labeled content, highlighting the need for alternative strategies to address challenges posed by AI-generated information.

研究动机与目标

• 评估AI署名标签是否改变AI生成政策信息的说服力。
• 在多个政策主题上测试三种标注条件（AI生成、人工撰写、无标签）。
• 在暴露于带标签信息前后，衡量政策支持及相关认知的变化。
• 评估在参与者特征（先前知识、AI经验、人口统计学特征）下的鲁棒性。

提出的方法

• 预注册的调查实验，样本量为1601名来自多样化美国样本的参与者。
• 随机分配到四项政策提案的AI标签、人工标签或无标签条件。
• 信息由GPT-4o生成，使用基于证据的说服技术进行事实准确性编辑。
• 结果变量包括政策支持的变化（主变量）、信心、准确性判断和分享意向，采用0–100的量表。
• 回归分析控制政策和干预前支持度，以比较不同标签条件。

实验结果

研究问题

• RQ1 将AI生成内容标注为AI、人工或无标签是否影响其在政策态度上的说服力？
• RQ2 在标签条件之间，感知信息准确性、对支持的信心或分享意向是否存在差异？
• RQ3 参与者特征（先前知识、AI经验、政治身份、教育、年龄）是否调节标注效应？
• RQ4 在多个政策领域中，标签效应是否具有稳健的一致性？

主要发现

• 总体上信息具有说服力，平均使政策支持度变化为9.74个百分点。
• AI标注组中有94.6%的参与者以及人工标注组中有89.3%的参与者相信分配的署名标签。
• AI标注、人工标注和无标签条件之间在说服力方面没有显著差异。
• 在对支持的信心、信息准确性判断或分享意向等方面，不同标签条件之间也未出现显著差异。