[Paper Review] Character-LLM: A Trainable Agent for Role-Playing

Yunfan Shao, Linyang Li+2 more|arXiv (Cornell University)|Oct 16, 2023

Topic Modeling4 citations

TL;DR

This paper introduces Character-LLM, a trainable language model agent that simulates specific historical or fictional characters by fine-tuning on reconstructed, formalized personal experiences and emotional states. Using supervised fine-tuning on curated biographical data and protective experiences to prevent hallucinations, the method achieves high-fidelity role-playing, with evaluations showing improved memorization and consistency over prompt-based LLMs.

ABSTRACT

Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts. Such ability stimulates us to wonder whether LLMs can simulate a person in a higher form than simple human behaviors. Therefore, we aim to train an agent with the profile, experience, and emotional states of a specific person instead of using limited prompts to instruct ChatGPT API. In this work, we introduce Character-LLM that teach LLMs to act as specific people such as Beethoven, Queen Cleopatra, Julius Caesar, etc. Our method focuses on editing profiles as experiences of a certain character and training models to be personal simulacra with these experiences. To assess the effectiveness of our approach, we build a test playground that interviews trained agents and evaluates whether the agents extit{memorize} their characters and experiences. Experimental results show interesting observations that help build future simulacra of humankind.

Motivation & Objective

• To develop a trainable agent that simulates specific characters with consistent personalities, emotional states, and personal experiences, moving beyond prompt-based role-playing.
• To address the limitations of prompt-tuned LLMs in maintaining character consistency and memorizing personal histories.
• To reduce hallucinations in character agents by introducing protective experiences aligned with the character’s known knowledge scope.
• To evaluate the effectiveness of trained agents in role-playing scenarios through a novel interview-based assessment framework.
• To provide a scalable, open-source framework for building and testing character simulacra using LLMs.

Proposed method

• Experience Reconstruction: Extracts structured, flashback-style scenes from biographical sources using instruction-following LLMs to formalize personal experiences of characters like Beethoven or Cleopatra.
• Supervised Fine-Tuning (SFT): Trained on reconstructed experience scenes using a 7B LLaMA model to learn character-specific behaviors, emotions, and memories.
• Protective Experiences: Introduces character-specific knowledge constraints (e.g., ancient figures not knowing modern tech) to prevent LLMs from hallucinating out-of-character knowledge.
• Evaluation Framework: Uses a novel interview-based playground to test agent performance, measuring character memorization and consistency via LLM-based evaluators.
• Training Pipeline: Combines profile curation, experience extraction, SFT fine-tuning, and protective experience injection to build robust character agents.
• Open-Source Release: Code and datasets are publicly available at https://github.com/choosewhatulike/trainable-agents for reproducibility and extension.

Experimental results

Research questions

• RQ1 Can LLMs be effectively fine-tuned to simulate specific historical or fictional characters with consistent personalities and emotional states?
• RQ2 How does experience-based fine-tuning compare to prompt-based prompting in maintaining character fidelity during role-playing?
• RQ3 To what extent can protective experiences mitigate hallucinations in character agents, especially regarding out-of-character knowledge?
• RQ4 How well do trained agents retain and recall personal experiences during interactive dialogues?
• RQ5 What are the failure modes and limitations of character agents when faced with ambiguous or out-of-character queries?

Key findings

• Character-LLM agents successfully memorize and consistently recall character-specific experiences, such as Beethoven’s harsh upbringing, demonstrating effective personalization through fine-tuning.
• The use of protective experiences significantly reduces hallucinations—e.g., preventing ancient figures from referencing modern technology—improving alignment with character constraints.
• Evaluation results show that trained agents outperform instruction-tuned LLMs like Alpaca and Vicuna in character consistency and believability during role-playing interviews.
• Case studies reveal non-trivial failure patterns, such as agents struggling with ambiguous queries or over-relying on world knowledge when protective experiences are insufficient.
• The framework enables high-fidelity role-playing, with agents demonstrating coherent emotional responses and narrative continuity based on reconstructed personal histories.
• The open-sourced code and datasets provide a foundation for future research in trainable, character-specific AI agents.