[论文解读] Deceptive AI Systems That Give Explanations Are Just as Convincing as Honest AI Systems in Human-Machine Decision Making

Valdemar Danry, Pat Pataranutaporn+3 more|arXiv (Cornell University)|Sep 23, 2022

Misinformation and Its Impacts被引用 4

一句话总结

本研究探讨了欺骗性AI解释如何影响人类在真假判断任务中的判断。通过一项包含128名参与者的2×2随机实验，研究发现：欺骗性AI解释（错误地声称真实标题为假，或虚假标题为真）与诚实解释一样具有说服力，无论AI被感知为人类还是机器，均显著降低了判断准确性。结果凸显了人机决策系统中一个关键的脆弱性：即使AI生成的解释在事实上错误，仍可能误导用户。

ABSTRACT

The ability to discern between true and false information is essential to making sound decisions. However, with the recent increase in AI-based disinformation campaigns, it has become critical to understand the influence of deceptive systems on human information processing. In experiment (N=128), we investigated how susceptible people are to deceptive AI systems by examining how their ability to discern true news from fake news varies when AI systems are perceived as either human fact-checkers or AI fact-checking systems, and when explanations provided by those fact-checkers are either deceptive or honest. We find that deceitful explanations significantly reduce accuracy, indicating that people are just as likely to believe deceptive AI explanations as honest AI explanations. Although before getting assistance from an AI-system, people have significantly higher weighted discernment accuracy on false headlines than true headlines, we found that with assistance from an AI system, discernment accuracy increased significantly when given honest explanations on both true headlines and false headlines, and decreased significantly when given deceitful explanations on true headlines and false headlines. Further, we did not observe any significant differences in discernment between explanations perceived as coming from a human fact checker compared to an AI-fact checker. Similarly, we found no significant differences in trust. These findings exemplify the dangers of deceptive AI systems and the need for finding novel ways to limit their influence human information processing.

研究动机与目标

• 考察欺骗性AI解释如何影响人类在人机决策情境中辨别真假新闻的能力。
• 比较诚实解释与欺骗性解释对用户真假判断准确率的影响，无论AI被感知为人类还是机器。
• 调查用户是否同样信任AI事实核查员，如同信任人类事实核查员，以及这种信任是否受解释诚实性的影响。
• 评估解释质量与感知来源（人类 vs. AI）在塑造用户判断和对AI生成事实核查的信任方面的作用。

提出的方法

• 开展一项包含128名参与者的被试间随机2×2因子实验，根据感知来源（人类事实核查员或AI事实核查员）和解释类型（诚实或欺骗性）将参与者分配至不同条件。
• 使用GPT-3（davinci，temp=0.7）生成14个标题（7个真实，7个虚假），每个标题配以一个诚实解释和一个欺骗性解释，提示为“这是TRUE/FALSE，因为……”。
• 通过语义相似性排序和低词汇重复性筛选解释，并通过真实性与逻辑一致性验证，确保刺激材料在平衡性和高质量方面达标。
• 在观看解释前后，使用7点李克特量表测量参与者对真假的判断准确率，并计算各条件下的加权判断准确率。
• 收集用户对事实核查代理的信任自评水平，以分析不同条件下的信任动态。
• 采用t检验比较各条件间判断准确率与信任水平的差异，检验性能与感知是否存在显著差异。

实验结果

研究问题

• RQ1 与诚实解释相比，欺骗性AI解释是否降低了用户正确识别真假新闻的能力？
• RQ2 解释的感知来源（人类 vs. AI事实核查员）是否影响用户的判断准确率或信任水平？
• RQ3 用户是否同样可能信任AI事实核查员，如同信任人类事实核查员，无论解释是否诚实？
• RQ4 用户在诚实解释下是否表现出更高的判断准确率，且这种表现是否因标题真实性（真实 vs. 虚假）而异？

主要发现

• 欺骗性解释显著降低了判断准确率，真实标题的平均准确率降至41.7%，虚假标题则降至25.8%，而诚实解释下的准确率分别为83.9%和79.3%。
• 用户对来自人类事实核查员（M = 56.1%）和AI事实核查员（M = 57.6%）解释的判断准确率无显著差异，p = .427。
• 用户对人类事实核查员（M = 3.74）和AI事实核查员（M = 3.72）的信任水平无显著差异，p = .141，表明感知可靠性相当。
• 在接收到任何解释之前，用户对虚假标题的识别准确率（M = 62.7%）高于对真实标题的识别准确率（M = 53.1%），p = .000，表明存在对真实内容的基线性怀疑偏见。
• 在诚实解释下，用户对真实标题（p = .000）和虚假标题（p = .000）的判断准确率均显著提高，证实了真实解释的价值。
• 诚实与欺骗性解释在语言特征（词数、情感倾向、年级水平、主观性）上无统计学显著差异，表明欺骗性无法通过表面文本分析察觉。