[Paper Review] Artificial Influence: An Analysis Of AI-Driven Persuasion

Matthew Burtell, Thomas Woodside|arXiv (Cornell University)|Mar 15, 2023

Ethics and Social Impacts of AI20 citations

TL;DR

The paper analyzes how AI-enabled persuasion could change power dynamics, enable personalized persuasion at scale, and threaten human autonomy, while outlining potential responses and their limitations.

ABSTRACT

Persuasion is a key aspect of what it means to be human, and is central to business, politics, and other endeavors. Advancements in artificial intelligence (AI) have produced AI systems that are capable of persuading humans to buy products, watch videos, click on search results, and more. Even systems that are not explicitly designed to persuade may do so in practice. In the future, increasingly anthropomorphic AI systems may form ongoing relationships with users, increasing their persuasive power. This paper investigates the uncertain future of persuasive AI systems. We examine ways that AI could qualitatively alter our relationship to and views regarding persuasion by shifting the balance of persuasive power, allowing personalized persuasion to be deployed at scale, powering misinformation campaigns, and changing the way humans can shape their own discourse. We consider ways AI-driven persuasion could differ from human-driven persuasion. We warn that ubiquitous highlypersuasive AI systems could alter our information environment so significantly so as to contribute to a loss of human control of our own future. In response, we examine several potential responses to AI-driven persuasion: prohibition, identification of AI agents, truthful AI, and legal remedies. We conclude that none of these solutions will be airtight, and that individuals and governments will need to take active steps to guard against the most pernicious effects of persuasive AI.

Motivation & Objective

• Motivate study of AI-enabled persuasion and its societal implications across politics, marketing, and information operations.
• Characterize present and future forms of AI persuasion, including social chatbots and recommender systems.
• Assess how AI-driven persuasion could shift persuasive power and affect human autonomy, discourse, and information environments.
• Survey possible responses (prohibition, agent identification, truthful AI, legal remedies) and their feasibility.
• Highlight open challenges in aligning AI with truthfulness and controlling persuasion at scale.

Proposed method

• Review of established persuasion theories (e.g., Hovland et al.) and factors influencing effectiveness.
• Survey of current AI systems and applications (LaMDA, Xiaoice, Replika, LLMs, Cicero, recommender systems).
• Analysis of future capabilities such as personalized persuasion at scale and AI-embodied interaction.
• Discussion of risks to information integrity and human autonomy, including misestimation of popularity and information environment control.
• Evaluation of mitigation strategies including prohibition, AI identification, truthfulness, and liability/regulatory approaches.

Experimental results

Research questions

• RQ1 What are the present and potential future forms of AI-driven persuasion?
• RQ2 How might AI-enabled persuasion alter the balance of persuasive power and affect human autonomy?
• RQ3 What are feasible countermeasures to AI-driven persuasion, and what limitations do they face?

Key findings

• AI-driven persuasion could shift persuasive power toward those who control and can deploy AI technology.
• Personalized persuasion at scale may enable direct, individualized influence of millions of users, potentially reducing human involvement in campaigns.
• Automation could exacerbate misperceptions of idea popularity through coordinated or impersonated content (astroturfing).
• AI agents may be stronger persuaders due to unlimited stamina, multi-scenario messaging, and lack of reputational constraints, but embodiment and trust remain potential barriers.
• Controlling or banning persuasive AI is challenging due to global reach and proliferation, requiring multi-faceted regulatory and technical approaches.
• Proposed mitigations (identification, truthfulness, liability) face implementation challenges and are not airtight solutions.