[Paper Review] The Effects of Embodiment and Personality Expression on Learning in LLM-based Educational Agents

Si̇nan Sonlu, Bennie Bendiksen+2 more|arXiv (Cornell University)|Jun 24, 2024

Education and Learning Interventions4 citations

TL;DR

This study investigates how personality expression and embodiment affect learning in LLM-based educational agents, using a 3D animated agent with GPT-3.5-driven dialogue and facial/body animations. Results show that high-extroversion/agreeableness agents are perceived as more engaging, though no significant differences in learning outcomes were found across model types, with all models rated positively overall.

ABSTRACT

This work investigates how personality expression and embodiment affect personality perception and learning in educational conversational agents. We extend an existing personality-driven conversational agent framework by integrating LLM-based conversation support tailored to an educational application. We describe a user study built on this system to evaluate two distinct personality styles: high extroversion and agreeableness and low extroversion and agreeableness. For each personality style, we assess three models: (1) a dialogue-only model that conveys personality through dialogue, (2) an animated human model that expresses personality solely through dialogue, and (3) an animated human model that expresses personality through both dialogue and body and facial animations. The results indicate that all models are positively perceived regarding both personality and learning outcomes. Models with high personality traits are perceived as more engaging than those with low personality traits. We provide a comprehensive quantitative and qualitative analysis of perceived personality traits, learning parameters, and user experiences based on participant ratings of the model types and personality styles, as well as users' responses to open-ended questions.

Motivation & Objective

• To examine the impact of personality expression and embodiment on learning outcomes in LLM-based educational agents.
• To evaluate how different personality styles—high vs. low extroversion and agreeableness—affect user perception and engagement.
• To compare three agent models: dialogue-only, animated without gestures, and animated with full gesture and facial animation.
• To assess whether embodiment and personality expression influence perceived learning, quality, and engagement in educational interactions.
• To explore the role of multimodal cues (dialogue, animation) in shaping user perception of agent personality and educational effectiveness.

Proposed method

• Extended a personality-driven conversational agent framework with GPT-3.5 for dialogue generation in an educational context.
• Designed two personality styles: high extroversion-agreeableness (energetic, warm) and low extroversion-agreeableness (reserved, formal).
• Implemented three agent models: (1) dialogue-only, (2) animated with dialogue only, and (3) animated with dialogue and synchronized gestures/facial expressions.
• Conducted a three-by-two independent-subjects user study with 60 participants, each interacting with one agent variant.
• Collected quantitative ratings on learning, quality, engagement, and perceived personality across all five FFM traits.
• Used open-ended questions and thematic analysis to explore qualitative user experiences and perception nuances.

Figure 1 : Different 3D agent models used in the study expressing high (left group) and low (right group) traits.

Experimental results

Research questions

• RQ1 RQ1. Is there an effect of personality style on perceived personality?
• RQ2 RQ2. Is there an effect of model type on perceived personality?
• RQ3 RQ3. Is there an effect of model type on learning outcomes?
• RQ4 RQ4. Is there a correlation between learning outcomes and personality perception?

Key findings

• Participants perceived agents with high extroversion and agreeableness as significantly more engaging than those with low traits, regardless of model type.
• All models were rated positively across all five Five-Factor Model personality dimensions, with high-trait agents receiving higher ratings in openness, conscientiousness, extroversion, agreeableness, and lower neuroticism.
• The embodied agent with full animation received higher engagement scores than the dialogue-only and animated-only models, though the difference was not statistically significant.
• No significant differences were found in perceived learning or quality across model types, indicating that embodiment alone did not improve learning outcomes in this study.
• Participants reported that the synthesized voice sounded unnatural, and some found responses too long or complex, suggesting limitations in current TTS and content adaptation.
• Thematic analysis revealed strong individual differences in user preferences, with some users favoring personality compatibility over model type, indicating potential for personalized agent selection.

(a) Model D