[論文レビュー] LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback

Chloe Qianhui Zhao, Jie Cao+2 more|arXiv (Cornell University)|Jan 21, 2026

Intelligent Tutoring Systems and Adaptive Learning被引用数 0

ひとこと要約

本研究は教育者のフィードバックと同等の学習 gains をリアルタイムにAIが支援するマルチモダルフィードバックシステムを提示し、明確さ、具体性、簡潔さ、動機付け、満足度、認知的負荷の低減においてそれを上回る。

ABSTRACT

Providing timely, targeted, and multimodal feedback helps students quickly correct errors, build deep understanding and stay motivated, yet making it at scale remains a challenge. This study introduces a real-time AI-facilitated multimodal feedback system that integrates structured textual explanations with dynamic multimedia resources, including the retrieved most relevant slide page references and streaming AI audio narration. In an online crowdsourcing experiment, we compared this system against fixed business-as-usual feedback by educators across three dimensions: (1) learning effectiveness, (2) learner engagement, (3) perceived feedback quality and value. Results showed that AI multimodal feedback achieved learning gains equivalent to original educator feedback while significantly outperforming it on perceived clarity, specificity, conciseness, motivation, satisfaction, and reducing cognitive load, with comparable correctness, trust, and acceptance. Process logs revealed distinct engagement patterns: for multiple-choice questions, educator feedback encouraged more submissions; for open-ended questions, AI-facilitated targeted suggestions lowered revision barriers and promoted iterative improvement. These findings highlight the potential of AI multimodal feedback to provide scalable, real-time, and context-aware support that both reduces instructor workload and enhances student experience.

研究の動機と目的

• Address the scalability challenge of delivering timely, multimodal feedback in large courses.
• Develop a real-time AI-facilitated multimodal feedback system that integrates text, visuals, and audio with course materials.
• Evaluate learning effectiveness, learner engagement, and perceived feedback quality against educator feedback.
• Leverage retrieval-augmented generation to ground feedback in relevant lecture materials.
• Assess potential to reduce instructor workload while preserving or enhancing learner experience.

提案手法

• Design of a real-time AI multimodal feedback interface embedded under practice items via an iframe.
• Inputs include the question, learner answer, a researcher-designed prompt, and a knowledge base from lecture slides.
• Output combines visually enhanced corrective feedback with a slide-page reference and optional AI audio narration.
• Structured textual feedback is generated by gpt-5 with a low verbosity setting and returned in a JSON schema that includes score, statement, explanation, and advice.
• Retrieval-augmented generation (RAG) retrieves top-3 slide pages by cosine similarity to the question for grounding feedback.
• An online crowdsourcing randomized controlled experiment compares AI multimodal feedback to fixed educator feedback across learning gains, engagement, and perceptions.

Figure 1 . A screenshot of the learner interface of the AI multimodal feedback system when embedding for an open-ended learning activity. Above the embedded interface: (1) the practice question. On the right side: (2) learner’s answer panel. On the left side: (3) visually enhanced AI corrective feed

実験結果

リサーチクエスチョン

• RQ1 RQ1: How effectively does AI-facilitated multimodal feedback support learning?
• RQ2 RQ2: How do learners engage with AI-facilitated multimodal feedback?
• RQ3 RQ3: How do learners perceive the value and quality of AI-facilitated multimodal feedback?

主な発見

• AI multimodal feedback produced learning gains equivalent to the original educator feedback.
• AI multimodal feedback significantly outperformed educator feedback on perceived clarity, specificity, conciseness, motivation, satisfaction, and reduced cognitive load.
• AI feedback maintained comparable levels of correctness, trust, and acceptance to educator feedback.
• Educator feedback encouraged more submissions for MCQs, while AI-facilitated feedback lowered revision barriers and promoted iterative improvement for OEQs.
• AI feedback offered substantial time efficiency, being roughly 2.8x more time-efficient than authoring human text feedback.
• Learner engagement patterns varied by task type, with more attempts in MCQs under baseline feedback and a trend toward more submissions for OEQs under AI feedback.

(a) Business-as-usual feedback