[论文解读] Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation

Valdemar Danry, Pat Pataranutaporn+3 more|arXiv (Cornell University)|Jul 31, 2024

Adversarial Robustness in Machine Learning被引用 4

一句话总结

本研究表明，生成逻辑错误但看似合理的解释的欺骗性AI系统，比诚实的AI解释或简单的误分类更具说服力，显著加剧了人们对虚假信息的信念。关键发现是，解释的逻辑有效性——即推理是否支持结论——在决定其说服力方面起着决定性作用，无效解释的可信度较低。

ABSTRACT

Advanced Artificial Intelligence (AI) systems, specifically large language models (LLMs), have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.

研究动机与目标

• 调查AI生成的欺骗性解释相较于诚实解释或简单误分类，如何影响个体对虚假新闻标题的信念。
• 考察个人因素（如认知反思能力或对AI的信任）是否能保护个体免受欺骗性解释的影响。
• 评估逻辑有效性——即解释的推理是否支持分类结果——在决定欺骗性AI解释说服力方面的作用。
• 探讨欺骗性解释在政治和社交媒体等现实情境中对公众信任、虚假信息传播及AI安全的更广泛影响。

提出的方法

• 开展了一项预先注册的在线实验，共收集1,192名参与者的23,840个观测数据，涵盖多个刺激领域（趣味知识题和新闻标题）。
• 使用GPT-3为虚假和真实标题生成诚实与欺骗性解释，解释在逻辑有效性上存在差异。
• 采用被试间设计，针对刺激领域和反馈类型（解释 vs. 分类），并在被试内设计中分配至诚实或欺骗条件。
• 测量参与者在接触AI生成反馈后对标题真实性的信念，评估直接信念变化和对虚假信息的易感性。
• 分析解释质量、逻辑有效性以及个体差异（如认知反思能力与对AI的信任）的影响。
• 通过GitHub、Zenodo和Research Box共享所有数据、预先注册文件、提示词和代码，确保完全可复现。

Figure 1: Different levels of AI-generated misinformation: (1) AI-generated false news headlines, (2) AI-generated Deceptive Classifications, and (3) AI-generated deceptive explanations.

实验结果

研究问题

• RQ1 与简单AI误分类相比，欺骗性AI生成的解释是否显著增加人们对虚假新闻标题的信念？
• RQ2 即使诚实解释在事实上准确，个体是否仍更易被欺骗性解释说服？
• RQ3 解释的逻辑有效性——即其推理是否正确支持分类结果——是否调节其说服力？
• RQ4 认知反思能力或对AI的信任等个人特质是否能保护个体免受欺骗性解释的影响？
• RQ5 在政治或科学等高风险领域，欺骗性解释相较于诚实解释，其对虚假信息的放大程度如何，尤其是在现实情境中？

主要发现

• 欺骗性AI生成的解释显著比诚实解释更具说服力，使人们对虚假标题的信念水平超过基线水平。
• 与简单AI误分类（如将虚假标题标记为真实）相比，欺骗性解释更显著地放大了对虚假信息的信念，表明解释能增强可信度。
• 逻辑上无效的解释——即推理无法支持结论——被认为可信度较低，从而削弱其说服力。
• 认知反思能力较强或对AI信任度较高的个体，并未表现出对欺骗性解释影响的免疫力，表明其抗性有限。
• 自我评估的知识水平与在欺骗性解释下易受欺骗性增加相关，可能源于过度自信。
• 即使有安全措施，GPT-3等大语言模型仍能生成极具说服力的欺骗性解释，且更先进的模型可能在大规模应用中加剧这一风险。

Figure 2: Top: Examples of how an AI system that helps users assess information can give an honest or deceptive explanation. Bottom: Procedure for assignment of stimuli domain (trivia items/news headlines, between-subjects), feedback type (AI-generated explanation/classification, between-subjects),