[论文解读] Harnessing Large Language Models to Enhance Self-Regulated Learning via Formative Feedback

Steffen Steinert, Karina E. Avila+3 more|arXiv (Cornell University)|Nov 23, 2023

Innovative Teaching and Learning Methods被引用 11

一句话总结

本文介绍 LEAP，一种开放获取平台，使用大型语言模型提供基于理论的形成性反馈以支持自我调节学习，提示设计围绕 SRL 原则及认知激活活动。

ABSTRACT

Effectively supporting students in mastering all facets of self-regulated learning is a central aim of teachers and educational researchers. Prior research could demonstrate that formative feedback is an effective way to support students during self-regulated learning (SRL). However, for formative feedback to be effective, it needs to be tailored to the learners, requiring information about their learning progress. In this work, we introduce LEAP, a novel platform that utilizes advanced large language models (LLMs), such as ChatGPT, to provide formative feedback to students. LEAP empowers teachers with the ability to effectively pre-prompt and assign tasks to the LLM, thereby stimulating students' cognitive and metacognitive processes and promoting self-regulated learning. We demonstrate that a systematic prompt design based on theoretical principles can provide a wide range of types of scaffolds to students, including sense-making, elaboration, self-explanation, partial task-solution scaffolds, as well as metacognitive and motivational scaffolds. In this way, we emphasize the critical importance of synchronizing educational technological advances with empirical research and theoretical frameworks.

研究动机与目标

• 通过定制化形成性反馈来激发并促进有效的自我调节学习（SRL）。
• 展示系统化提示设计如何促进认知、元认知和动机支架。
• 提供一个对教师友好的工具，在控制输出质量的同时减少编程需求。

提出的方法

• 介绍 LEAP，一个基于网络的平台，用于为 LLMs 创建提示和任务。
• 描述基于 SRL 与形成性反馈理论的三部分提示设计（Condition、Consequence、Refinement）。
• 解释实现细节：Flask 后端、MariaDB 数据库，以及使用 OpenAI ChatGPT API。
• 演示生成认知激活性反馈（sense-making、elaboration、self-explanation）以及元认知/动机支架的提示。
• 展示教师如何为学生反馈选择复杂度等级（low、middle、high）。
• 讨论实现受控任务创建、匿名分发给学生以及用于研究的数据记录的架构。

Figure 1: Systematic prompt generation in three parts based on theoretical principles.

实验结果

研究问题

• RQ1 如何提示 LLMs 以提供支持认知、元认知和动机的形成性反馈用于 SRL？
• RQ2 通过预定义提示，面向教师的平台能否可靠地产生多样的支架（sense-making、elaboration、self-explanation、partial task solution、元认知与动机提示）？
• RQ3 结构化提示在降低幻觉与偏见、同时提升 SRL 任务学习成果中的作用是什么？

主要发现

• 系统性、基于理论的三部分提示设计使 LLMs 能够提供多样的认知支架，而不向学生暴露提示细节。
• 更复杂的提示往往会带来更丰富的反馈，尤其是在高级的 ChatGPT 版本中。
• LEAP 使教师能够进行预提示并分配任务，在促进 SRL 的同时减少学生的提示撰写需求。
• 该平台在 SRL 维度（认知、元认知、动机）上提供实时、个性化的形成性反馈。
• LEAP 的架构允许受控、隐私保护的任务分发及研究用的互动数据收集。

Figure 2: The LEAP window displays an example of how to assign a task to students. This process includes several steps: selecting the GPT version, naming the task, formulating the question for students, setting instructions for ChatGPT, and providing the correct answer.