[Paper Review] Harnessing Large Language Models to Enhance Self-Regulated Learning via Formative Feedback

Steffen Steinert, Karina E. Avila+3 more|arXiv (Cornell University)|Nov 23, 2023

Innovative Teaching and Learning Methods11 citations

TL;DR

The paper presents LEAP, an open-access platform that uses large language models to provide theory-driven formative feedback to support self-regulated learning, with prompts designed around SRL principles and cognitively activating activities.

ABSTRACT

Effectively supporting students in mastering all facets of self-regulated learning is a central aim of teachers and educational researchers. Prior research could demonstrate that formative feedback is an effective way to support students during self-regulated learning (SRL). However, for formative feedback to be effective, it needs to be tailored to the learners, requiring information about their learning progress. In this work, we introduce LEAP, a novel platform that utilizes advanced large language models (LLMs), such as ChatGPT, to provide formative feedback to students. LEAP empowers teachers with the ability to effectively pre-prompt and assign tasks to the LLM, thereby stimulating students' cognitive and metacognitive processes and promoting self-regulated learning. We demonstrate that a systematic prompt design based on theoretical principles can provide a wide range of types of scaffolds to students, including sense-making, elaboration, self-explanation, partial task-solution scaffolds, as well as metacognitive and motivational scaffolds. In this way, we emphasize the critical importance of synchronizing educational technological advances with empirical research and theoretical frameworks.

Motivation & Objective

• Motivate and enable effective self-regulated learning (SRL) through tailored formative feedback.
• Show how systematic prompt design can stimulate cognitive, metacognitive, and motivational scaffolds.
• Provide a teacher-friendly tool that reduces the need for programming while controlling output quality.

Proposed method

• Introduce LEAP, a web-based platform for creating prompts and tasks for LLMs.
• Describe three-part prompt design (Condition, Consequence, Refinement) grounded in SRL and formative feedback theories.
• Explain implementation details: Flask backend, MariaDB database, and usage of OpenAI ChatGPT APIs.
• Demonstrate prompts that generate cognitively activating feedback (sense-making, elaboration, self-explanation) and metacognitive/motivational scaffolds.
• Show how teachers can select complexity levels for student feedback (low, middle, high).
• Discuss architecture that enables controlled task creation, anonymous distribution to students, and data logging for research.

Figure 1: Systematic prompt generation in three parts based on theoretical principles.

Experimental results

Research questions

• RQ1 How can LLMs be prompted to provide formative feedback that supports cognition, metacognition, and motivation in SRL?
• RQ2 Can a teacher-friendly platform reliably generate diverse scaffolds (sense-making, elaboration, self-explanation, partial task solution, metacognitive and motivational prompts) through predefined prompts?
• RQ3 What is the role of structured prompting in reducing hallucinations and bias while enhancing learning outcomes in SRL tasks?

Key findings

• A systematic, theory-based three-part prompt design enables LLMs to provide varied cognitive scaffolds without exposing students to prompt details.
• More complex prompts tend to elicit richer feedback, especially with advanced ChatGPT versions.
• LEAP enables teachers to pre-prompt and assign tasks, fostering SRL while reducing prompt-writing requirements for students.
• The platform supports real-time, personalized formative feedback across SRL dimensions (cognition, metacognition, motivation).
• LEAP’s architecture allows for controlled, privacy-preserving delivery of tasks and collection of interaction data for research.

Figure 2: The LEAP window displays an example of how to assign a task to students. This process includes several steps: selecting the GPT version, naming the task, formulating the question for students, setting instructions for ChatGPT, and providing the correct answer.