[Paper Review] MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education

Murong Yue, Lyu, Wenhan+3 more|arXiv (Cornell University)|Apr 10, 2024

Topic Modeling4 citations

TL;DR

MathVC introduces the first LLM-powered virtual classroom with multiple LLM-simulated student characters to enable human students to practice mathematical modeling (MM) through authentic, collaborative discussions. By integrating domain-specific knowledge, symbolic thought schemas, and a meta-planner, MathVC achieves characteristic and conversational procedural alignment, significantly improving the realism and educational value of simulated MM group work.

ABSTRACT

Collaborative problem solving (CPS) is essential in mathematics education, fostering deeper learning through the exchange of ideas. Yet, classrooms often lack the resources, time, and peer dynamics needed to sustain productive CPS. Recent advancements in Large Language Models (LLMs) offer a promising avenue to enhance CPS in mathematical education. We designed and developed MathVC, a multi-persona LLM simulated virtual classroom platform to facilitate CPS in mathematics. MathVC combines a meta planning controller that monitors CPS stages-sense-making, team organization, planning, execution, validation, and predicts the next speaker, with a persona simulation stack that encodes mathematical thinking via a task schema and error-injected persona schemas seeded from teacher-specified misconceptions. We evaluated MathVC with 14 U.S. middle schoolers. Students reported constructive interaction and reaching shared solutions, describing gains in engagement, motivation, and confidence through diverse perspectives, immediate scaffolding, and human-like fallibility. Our findings also provide insights into simulating peers via LLM-based technologies for collaboration to support learning.

Motivation & Objective

• To address the lack of access to collaborative mathematical modeling (MM) practice in under-resourced educational settings.
• To simulate authentic middle-school student behaviors in MM group discussions using LLMs, overcoming limitations of vanilla prompt-based character simulation.
• To ensure LLM-simulated students evolve their understanding iteratively, reflecting real student learning processes.
• To maintain a coherent, stage-based conversational flow that mirrors real MM discussion procedures.
• To develop a scalable, teacher-free platform for enhancing MM skills through interactive, multi-agent virtual classrooms.

Proposed method

• Integrating domain-specific mathematical modeling knowledge into LLM prompting to guide realistic problem-solving behavior.
• Defining symbolic thought schemas for each LLM character, encoding their unique traits (e.g., math proficiency, reasoning style) to ensure characteristic alignment.
• Designing a meta-planner at the platform level to orchestrate the conversational procedure across characters, ensuring progression through stages like problem understanding, task division, and solution planning.
• Using a two-stage LLM inference pipeline: first generating dialogue acts (actions) based on schema and conversation context, then generating natural language responses grounded in those actions.
• Employing structured prompting with explicit variable tracking (e.g., 'tomatos_servings': 200) to maintain consistency and detect inconsistencies across agents.
• Validating alignment through ablation studies and qualitative analysis of simulated dialogues, comparing against real student discussion patterns.

Figure 1: An example demonstrating the application of MathVC , where students are presented a math problem (optionally with accompanied data) and engage in effective discussions on math modeling. Alice, Bob, and Charlie 3 3 3 We use the three names as “placeholders” following the conversion in scien

Experimental results

Research questions

• RQ1 Can LLM-simulated student characters with distinct traits realistically model the iterative, evolving nature of middle-school students’ mathematical modeling processes?
• RQ2 To what extent does the integration of symbolic thought schemas improve characteristic alignment in LLM-based student simulations?
• RQ3 How effective is the meta-planner in maintaining conversational procedural alignment across multiple LLM agents during collaborative problem-solving?
• RQ4 Does the proposed framework produce dialogues that are perceptually indistinguishable from real student MM discussions in structure and content?
• RQ5 What impact do the proposed components (schema, meta-planner, domain knowledge) have on the realism and educational fidelity of the virtual classroom?

Key findings

• The integration of domain-specific mathematical modeling knowledge significantly improved the realism of LLM-generated solutions and reasoning paths.
• Symbolic thought schemas enabled consistent and trait-aligned behavior across LLM characters, preventing premature or overly accurate solutions.
• The meta-planner successfully guided the conversation through structured stages (e.g., problem understanding, task division), mimicking authentic student group dynamics.
• Ablation studies confirmed that removing any of the three core components (schema, meta-planner, domain knowledge) degraded the quality and authenticity of the simulated discussions.
• The system produced dialogues with high procedural coherence and variable consistency, demonstrating the ability to simulate realistic collaborative problem-solving in MM.
• The platform enables human students to engage in meaningful, scaffolded practice of MM skills without requiring real-time teacher supervision.

Figure 2: Example task and character schema. The LLM edits the correct value 16 into 15 in the task schema to simulate the initial mistake made by the simulated student Charlie.