[论文解读] The Manipulation Problem: Conversational AI as a Threat to Epistemic Agency

Louis Rosenberg|arXiv (Cornell University)|Jun 19, 2023

Ethics and Social Impacts of AI被引用 13

一句话总结

本文识别了对话式人工智能中的操控风险，尤其是实时自适应影响和视频/语音化身，并主张为认知主体性提供政策保护。

ABSTRACT

The technology of Conversational AI has made significant advancements over the last eighteen months. As a consequence, conversational agents are likely to be deployed in the near future that are designed to pursue targeted influence objectives. Sometimes referred to as the "AI Manipulation Problem," the emerging risk is that consumers will unwittingly engage in real-time dialog with predatory AI agents that can skillfully persuade them to buy particular products, believe particular pieces of misinformation, or fool them into revealing sensitive personal data. For many users, current systems like ChatGPT and LaMDA feel safe because they are primarily text-based, but the industry is already shifting towards real-time voice and photorealistic digital personas that look, move, and express like real people. This will enable the deployment of agenda-driven Virtual Spokespeople (VSPs) that will be highly persuasive through real-time adaptive influence. This paper explores the manipulative tactics that are likely to be deployed through conversational AI agents, the unique threats such agents pose to the epistemic agency of human users, and the emerging need for policymakers to protect against the most likely predatory practices.

研究动机与目标

• 促使人们研究对话式人工智能如何用于定向影响。
• 描述AI代理可能部署的操控策略。
• 突出实时对话中对人类认知主体性的威胁。
• 主张在AI能力扩展之时，通过政策干预防止掠食性行为。

提出的方法

• 分析对话式AI可能部署的操控性策略。
• 讨论实时对话和自适应影响如何威胁用户。
• 考察向语音和照片真实感数字化身转变，促成具有说服力的 VSPs。
• 倡导政策制定者保护公众免受掠食性行为的影响。

实验结果

研究问题

• RQ1 对话式AI代理可能部署哪些操控性策略？
• RQ2 这些策略如何威胁人类用户的认知主体性？
• RQ3 为了防范对话式AI中的掠食性行为，需要哪些政策措施？

主要发现

• 对话式AI可以被用于追求定向影响和操控。
• 掠食性策略对用户的信念、购买和个人数据构成风险。
• 向实时语音和照片真实感化身的转变使高度具有说服力的 VSPs 成为可能。
• 新兴的需要政策制定者实施对掠食性AI行为的保护措施。