[Paper Review] CGMI: Configurable General Multi-Agent Interaction Framework

Shi Jinxin, Jiabao Zhao+4 more|arXiv (Cornell University)|Aug 24, 2023

Topic Modeling5 citations

TL;DR

CGMI proposes a configurable multi-agent framework that enhances LLM-based agents with a tree-structured persona model and ACT*-inspired cognitive architecture featuring a skill library, enabling stable, human-like interactions in complex social simulations. Evaluated in a classroom teaching scenario, CGMI successfully replicated realistic teaching dynamics, including role-specific behavior, student engagement variation, and adaptive questioning, demonstrating improved interaction rationality and stability over random or unstructured methods.

ABSTRACT

Benefiting from the powerful capabilities of large language models (LLMs), agents based on LLMs have shown the potential to address domain-specific tasks and emulate human behaviors. However, the content generated by these agents remains somewhat superficial, owing to their limited domain expertise and the absence of an effective cognitive architecture. To address this, we present the Configurable General Multi-Agent Interaction (CGMI) framework, designed to replicate human interactions in real-world scenarios. Specifically, we propose a tree-structured methodology for the assignment, detection, and maintenance of agent personality. Additionally, we designed a cognitive architecture equipped with a skill library based on the ACT* model, which contains memory, reflection, and planning modules. We have also integrated general agents to augment the virtual environment's realism. Using the CGMI framework, we simulated numerous classroom interactions between teacher and students. The experiments indicate that aspects such as the teaching methodology, curriculum, and student performance closely mirror real classroom settings. We will open source our work.

Motivation & Objective

• To address the limitations of LLM-based agents in maintaining consistent roles and deep domain expertise during long, complex interactions.
• To improve the realism and stability of multi-agent simulations by introducing a cognitive architecture with declarative and procedural memory retrieval.
• To design a tree-structured persona model that efficiently assigns, detects, and maintains agent personalities under memory constraints.
• To enhance scenario authenticity by integrating general agents that support role agents in task-specific collaboration.
• To validate the framework in a realistic classroom teaching scenario, simulating teacher-student interactions with human-like dynamics.

Proposed method

• Employing a tree-structured persona model to encode and maintain agent personality traits, reducing memory pressure and improving consistency across dialogues.
• Designing a cognitive architecture inspired by ACT*, incorporating memory, reflection, and planning modules with skill library retrieval.
• Using Chain-of-Thought (CoT) and Chain-of-Action (CoA) to extract and store declarative and procedural memories from working memory.
• Integrating general agents (e.g., assistant, supervisory agents) to support role agents in scenario-specific tasks and reduce cognitive load.
• Binding role agents (teacher, students) with general agents to enable dynamic, context-aware interaction logic in the classroom simulation.
• Applying personality-informed decision-making for interaction logic, such as willingness-to-speak and selective questioning, based on agent traits.

Experimental results

Research questions

• RQ1 How can LLM-based agents maintain stable, consistent personalities across long multi-turn dialogues in complex social simulations?
• RQ2 To what extent can a skill library and cognitive architecture improve the depth and domain-specific reasoning of LLM-based agents?
• RQ3 How does personality-informed interaction logic affect the realism and rationality of agent interactions in a classroom setting?
• RQ4 Can the integration of general agents enhance the authenticity and functionality of role-based multi-agent systems?
• RQ5 How well does the CGMI framework replicate real-world classroom teaching dynamics, including teaching methodology and student engagement patterns?

Key findings

• The personality-informed interaction logic significantly improved the rationality of student engagement, with high-willingness students like Emily being selected 9 times compared to only 3 times under random selection.
• The answer willingness judgment agent recommended students based on traits and classroom context, resulting in John being selected 4 times, Emily 9 times, Ryan 6 times, Samantha 1 time, and Ying Zheng 8 times over a full lesson.
• Random selection produced inconsistent results, such as selecting the introverted Samantha 6 times while Emily, the most expressive, was selected only 3 times, undermining scenario realism.
• The teacher’s selective questioning, guided by teaching plans and student status, was successfully simulated, with Ying Zheng being specifically targeted to explore multiple solutions, aligning with instructional design.
• The framework successfully simulated realistic teaching dynamics, including curriculum progression, teaching methodology, and student performance variation, closely mirroring real classroom behavior.
• The integration of general agents reduced the cognitive burden on role agents and enabled more natural, scenario-adapted collaboration, enhancing overall simulation fidelity.