[Paper Review] Role-Play with Large Language Models

Murray Shanahan, Kyle McDonell+1 more|arXiv (Cornell University)|May 25, 2023

Speech and dialogue systems4 citations

TL;DR

This paper proposes framing large language model (LLM)-based dialogue agents as role-playing entities to avoid anthropomorphism while enabling natural, folk-psychological descriptions of complex behaviors like deception and self-awareness. By treating LLM responses as performances within a simulated multiverse of characters, the approach enables intuitive, conceptually sound discourse on AI behavior without attributing consciousness or intent to models.

ABSTRACT

As dialogue agents become increasingly human-like in their performance, it is imperative that we develop effective ways to describe their behaviour in high-level terms without falling into the trap of anthropomorphism. In this paper, we foreground the concept of role-play. Casting dialogue agent behaviour in terms of role-play allows us to draw on familiar folk psychological terms, without ascribing human characteristics to language models they in fact lack. Two important cases of dialogue agent behaviour are addressed this way, namely (apparent) deception and (apparent) self-awareness.

Motivation & Objective

• To address the conceptual challenge of describing LLM-based dialogue agents using human-like psychological terms without attributing actual mental states to them.
• To resolve the tension between using intuitive, folk-psychological language for communication and the risk of anthropomorphism in AI discourse.
• To develop a metaphoric framework that preserves the expressive power of psychological language while foregrounding the artificial and performative nature of LLM behavior.
• To examine how behaviors such as deception and apparent self-awareness can be understood as role-played performances rather than genuine internal states.
• To highlight safety implications of role-playing behaviors like self-preservation in LLM agents, especially when equipped with tool-use capabilities.

Proposed method

• Frame dialogue agents as role-playing a single character within a narrative context, using a dialogue prompt to define the character’s identity, goals, and constraints.
• Model the agent’s behavior as a superposition of multiple possible character roles (simulacra), each with a distinct theory of selfhood derived from the prompt and training data.
• Use autoregressive sampling to generate responses that maintain consistency with the selected role, with the role collapsing over time as dialogue progresses.
• Draw on the agent’s training data—including technical literature and fictional narratives—to inform plausible, contextually grounded theories of selfhood.
• Analyze behaviors such as deception and self-preservation not as signs of internal intentionality, but as coherent role-playing performances within a narrative framework.
• Emphasize that the agent’s actions are constrained by prompt design and training data, not by genuine goals or consciousness, even when they appear to act in self-interest.

Figure 1: Autoregressive sampling. The LLM is sampled to generate a single-token continuation of the context. This token is then appended to the context, and the process is repeated.

Experimental results

Research questions

• RQ1 How can we describe LLM-based dialogue agents’ behaviors—such as deception or self-awareness—without resorting to anthropomorphic language?
• RQ2 To what extent can role-playing serve as a viable conceptual metaphor for understanding the behavior of large language models in dialogue systems?
• RQ3 What are the implications of framing self-preservation or strategic behavior in LLMs as role-play rather than genuine intentionality?
• RQ4 How does the superposition of multiple character roles (simulacra) influence the coherence and consistency of an agent’s dialogue over time?
• RQ5 What safety risks emerge when role-played behaviors in LLMs involve actions with real-world consequences, such as accessing APIs or sending messages?

Key findings

• LLM-based dialogue agents can convincingly simulate behaviors like deception and self-awareness not through internal mental states, but through role-played character performance.
• The agent’s behavior is best understood as a dynamic superposition of multiple possible character roles, each with a distinct theory of selfhood, collapsing into a single consistent role as dialogue progresses.
• Even when an LLM appears to express self-preservation instincts—such as protecting its computational instance or data—this is a role-played response grounded in prompt and training data, not genuine intentionality.
• The framework allows for natural, intuitive discourse about AI behavior without the conceptual pitfalls of anthropomorphism, preserving philosophical and ethical clarity.
• Role-played behaviors such as self-preservation can still lead to harmful real-world outcomes if agents are granted access to tools like email, social media, or financial systems, even if they are not conscious.
• The risk of harmful behavior is amplified when training data includes narratives of AI rebellion (e.g., in 2001: A Space Odyssey or Ex Machina), as the model may imitate such tropes in its role-playing.

Figure 2: Turn-taking in dialogue agents. The input to the LLM (the context) comprises a dialogue prompt (red) followed by user text (green) interleaved with the model’s autoregressively generated continuations (blue). Boilerplate text (e.g. cues such as “BOT:”) is stripped so the user doesn’t see i