Skip to main content
QUICK REVIEW

[论文解读] LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback

Chloe Qianhui Zhao, Jie Cao|arXiv (Cornell University)|Jan 21, 2026
Intelligent Tutoring Systems and Adaptive LearningComputer Science被引用 0
一句话总结

研究提出一个实时的AI辅助多模态反馈系统,学习增益方面与教育者反馈相当,在感知清晰度、具体性、简练性、动机、满意度以及认知负荷降低方面超越教育者反馈。

ABSTRACT

Providing timely, targeted, and multimodal feedback helps students quickly correct errors, build deep understanding and stay motivated, yet making it at scale remains a challenge. This study introduces a real-time AI-facilitated multimodal feedback system that integrates structured textual explanations with dynamic multimedia resources, including the retrieved most relevant slide page references and streaming AI audio narration. In an online crowdsourcing experiment, we compared this system against fixed business-as-usual feedback by educators across three dimensions: (1) learning effectiveness, (2) learner engagement, (3) perceived feedback quality and value. Results showed that AI multimodal feedback achieved learning gains equivalent to original educator feedback while significantly outperforming it on perceived clarity, specificity, conciseness, motivation, satisfaction, and reducing cognitive load, with comparable correctness, trust, and acceptance. Process logs revealed distinct engagement patterns: for multiple-choice questions, educator feedback encouraged more submissions; for open-ended questions, AI-facilitated targeted suggestions lowered revision barriers and promoted iterative improvement. These findings highlight the potential of AI multimodal feedback to provide scalable, real-time, and context-aware support that both reduces instructor workload and enhances student experience.

研究动机与目标

  • 解决在大规模课程中提供及时、多模态反馈的可扩展性挑战。
  • 开发一个实时的AI辅助多模态反馈系统,将文本、视觉和音频与课程材料整合。
  • 以教育者反馈为对比,评估学习效果、学习者参与度与反馈质量的感知。
  • 利用检索增强生成将反馈 grounding 于相关讲座材料。
  • 评估在保持或提升学习体验的同时,降低教师工作量的潜力。

提出的方法

  • 在练习题下方通过 iframe 嵌入的实时AI多模态反馈界面设计。
  • 输入包括问题、学习者答案、研究者设计的提示,以及来自讲义幻灯片的知识库。
  • 输出将视觉增强的纠错反馈、幻灯片页引用和可选的AI音频解说结合起来。
  • 由 gpt-5 以较低冗长度设置生成结构化文本反馈,并以 JSON 架构返回,包含 score、statement、explanation 与 advice。
  • 检索增强生成(RAG)按与问题的余弦相似度检索前3个幻灯片页用于对反馈的 grounding。
  • 在线众包随机对照实验比较AI多模态反馈与固定教育者反馈在学习增益、参与度和感知方面的差异。
Figure 1 . A screenshot of the learner interface of the AI multimodal feedback system when embedding for an open-ended learning activity. Above the embedded interface: (1) the practice question. On the right side: (2) learner’s answer panel. On the left side: (3) visually enhanced AI corrective feed
Figure 1 . A screenshot of the learner interface of the AI multimodal feedback system when embedding for an open-ended learning activity. Above the embedded interface: (1) the practice question. On the right side: (2) learner’s answer panel. On the left side: (3) visually enhanced AI corrective feed

实验结果

研究问题

  • RQ1RQ1: AI辅助的多模态反馈在多大程度上支持学习?
  • RQ2RQ2: 学习者如何参与AI辅助的多模态反馈?
  • RQ3RQ3: 学习者如何感知AI辅助的多模态反馈的价值和质量?

主要发现

  • AI多模态反馈产生的学习增益与原始教育者反馈相当。
  • AI多模态反馈在感知清晰度、具体性、简练性、动机、满意度以及降低认知负荷方面显著优于教育者反馈。
  • AI反馈在正确性、信任度和接受度方面与教育者反馈保持相当水平。
  • 教育者反馈鼓励对MCQ提交更多,而AI辅助反馈降低修订壁垒并促成OEQ的迭代改进。
  • AI反馈在时间效率方面具有显著优势,大约比撰写人工文本反馈节省2.8倍时间。
  • 学习者的参与模式因任务类型而异,基线反馈下MCQ有更多尝试;在AI反馈下OEQ呈现更多提交趋势。
(a) Business-as-usual feedback
(a) Business-as-usual feedback

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。