[Paper Review] Exploring Learners' Expectations and Engagement When Collaborating with Constructively Controversial Peer Agents

Thitaree Tanprasert, Young‐Ho Kim+2 more|arXiv (Cornell University)|Jan 20, 2026

Innovative Teaching and Learning Methods0 citations

TL;DR

The paper investigates how regulated versus unregulated CC-based behaviors and transparency of agent design affect learner engagement, sense of agency, and perceptions when collaborating with LLM-based peer agents in asynchronous settings.

ABSTRACT

Peer agents can supplement real-time collaborative learning in asynchronous online courses. Constructive Controversy (CC) theory suggests that humans deepen their understanding of a topic by confronting and resolving controversies. This study explores whether CC's benefits apply to LLM-based peer agents, focusing on the impact of agents' disputatious behaviors and disclosure of agents' behavior designs on the learning process. In our mixed-method study (n=144), we compare LLMs that follow detailed CC guidelines (regulated) to those guided by broader goals (unregulated) and examine the effects of disclosing the agents' design to users (transparent vs. opaque). Findings show that learners' values influence their agent interaction: those valuing control appreciate unregulated agents' willingness to cease push-back upon request, while those valuing intellectual challenges favor regulated agents for stimulating creativity. Additionally, design transparency lowers learners' perception of agents' abilities. Our findings lay the foundation for designing effective collaborative peer agents in isolated educational settings.

Motivation & Objective

• Examine whether CC-inspired behaviors in LLM-based peer agents enhance learning in asynchronous collaboration.
• Investigate how regulated versus unregulated CC behavior affects engagement, agency, and collaboration dynamics.
• Assess how disclosure of the agent’s behavior design (transparent vs opaque) moderates these effects.
• Identify learner value profiles and strategies in response to CC-based peer agents.
• Provide design insights for effective LLM-powered collaborative peer agents in isolated educational contexts.

Proposed method

• Conduct a 2 x 2 mixed factorial experiment with 144 undergraduates interacting with CC-based peer agents during argumentative task rounds.
• Compare unregulated (high-level CC guidelines, adaptable by the LLM) and regulated (strict CC guidelines, moderated) behaviors.
• Manipulate transparency of agent design (transparent vs opaque) as a between-subject factor.
• Use a combination of conversational analysis, task metrics (turn count, word count, completion time, argument quality) and self-reports of engagement and sense of agency.
• Collect open-ended surveys to capture qualitative insights on learner values and expectations (Sub-RQ1).

Figure 1. A summary diagram of the study procedure. After watching the experiment demo video, each participant completes two rounds of the task for the two behavior mechanisms (regulated and unregulated) with a 5-minute break in between. The dashed-border box shows the steps within each round. Note

Experimental results

Research questions

• RQ1 Sub-RQ1: What are learners’ values and expectations that influence their perception of and interaction with CC-based peer agents?
• RQ2 Sub-RQ2: How do regulated and unregulated CC behaviors affect engagement, sense of agency, and collaboration dynamics?
• RQ3 Sub-RQ3: How does disclosure or non-disclosure of the agent’s behavior designs moderate these effects?

Key findings

• Learners cluster into Efficiency-Driven and Curiosity-Driven profiles, linking to different collaborative strategies and preferences for CC behaviors.
• Disclosing the agent’s behavior design significantly lowers learners’ perception of the agent’s abilities across the board.
• Regulated CC behavior tends to stimulate creativity and provide more challenging engagement, while unregulated behavior offers flexibility aligned with learner control.
• Learners’ values influence their preferred balance between cooperation and contradiction in agent interactions.
• Two-agent design patterns (regulated vs unregulated) shape engagement, sense of agency, and collaboration dynamics in distinct ways.

Figure 2. This figure shows the experiment interface (a). The screen is divided into three columns. The leftmost column is the activity instruction, including the task scenario, the activity steps, and the argument grading rubrics. The middle column is the chat window, where the participant has a co