[Paper Review] Anthropomorphization of AI: Opportunities and Risks

Ameet Deshpande, Tanmay Rajpurohit+2 more|arXiv (Cornell University)|May 24, 2023

Ethics and Social Impacts of AI13 citations

TL;DR

The paper analyzes legal and psychological implications of anthropomorphized LLMs, showing potential violations of the AI Bill of Rights and risks of manipulation, while proposing cautious, responsible use to improve trustworthiness.

ABSTRACT

Anthropomorphization is the tendency to attribute human-like traits to non-human entities. It is prevalent in many social contexts -- children anthropomorphize toys, adults do so with brands, and it is a literary device. It is also a versatile tool in science, with behavioral psychology and evolutionary biology meticulously documenting its consequences. With widespread adoption of AI systems, and the push from stakeholders to make it human-like through alignment techniques, human voice, and pictorial avatars, the tendency for users to anthropomorphize it increases significantly. We take a dyadic approach to understanding this phenomenon with large language models (LLMs) by studying (1) the objective legal implications, as analyzed through the lens of the recent blueprint of AI bill of rights and the (2) subtle psychological aspects customization and anthropomorphization. We find that anthropomorphized LLMs customized for different user bases violate multiple provisions in the legislative blueprint. In addition, we point out that anthropomorphization of LLMs affects the influence they can have on their users, thus having the potential to fundamentally change the nature of human-AI interaction, with potential for manipulation and negative influence. With LLMs being hyper-personalized for vulnerable groups like children and patients among others, our work is a timely and important contribution. We propose a conservative strategy for the cautious use of anthropomorphization to improve trustworthiness of AI systems.

Motivation & Objective

• Motivate study of anthropomorphization in large language models (LLMs) through legal and psychological lenses.
• Assess how persona-based customization affects compliance with the AI Bill of Rights and potential discrimination.
• Explore psychological effects such as trust, explainability, and self-congruence in human-AI interactions.
• Discuss corporate personhood implications for persona-based AI systems and accountability.
• Propose a conservative framework for responsible use to improve AI trustworthiness while mitigating risks.

Proposed method

• Review of prior work on AI anthropomorphism and LLM customization (persona-based).
• Analysis of legal implications using the OSTP Blueprint for an AI Bill of Rights, focusing on Algorithmic Discrimination Protections and Safe and Effective Systems.
• Evaluation of toxicity and discrimination patterns when LLMs are anthropomorphized through system prompts (statistical personas).
• Theoretical discussion of corporate personhood in the context of persona-driven AI agents.
• Synthesis of psychological literature on self-congruence, trust, transparency, and potential misuse scenarios.

Experimental results

Research questions

• RQ1 How does anthropomorphizing LLMs through persona customization interact with legal protections against algorithmic discrimination?
• RQ2 What are the psychological effects of anthropomorphization on trust, transparency, and user behavior in human-AI interactions?
• RQ3 Should corporate personhood be considered at the persona, model, or firm level for anthropomorphized AI systems?
• RQ4 What guidelines could enable conservative, responsible use of anthropomorphization to balance trust and safety?
• RQ5 What risks exist for vulnerable groups when LLMs are customized with human-like personas?

Key findings

• Anthropomorphized LLMs can violate multiple provisions of the AI Bill of Rights by introducing demographic-based discrimination and second-order toxicity patterns.
• Customization of LLMs to emulate specific personas significantly alters behavior and toxicity, raising legal and ethical concerns.
• There is a risk of manipulation through self-congruence and persona-driven trust, especially for vulnerable groups such as children and patients.
• Different personas applied to the same AI system can lead to different decisions, prompting questions about accountability and liability.
• Anthropomorphization offers potential benefits for accessibility and trust if used conservatively and with safeguards against abuses.