[论文解读] Extending the Hint Factory for the assistance dilemma: A novel, data-driven HelpNeed Predictor for proactive problem-solving help

Mehak Maniktala, Christa Cody+4 more|arXiv (Cornell University)|Oct 8, 2020

Intelligent Tutoring Systems and Adaptive Learning参考文献 57被引用 6

一句话总结

本文提出了一种数据驱动的 HelpNeed 预测器，用于在命题逻辑等结构良好、开放式的问题解决领域中识别低效的问题解决步骤，扩展了 Hint Factory 框架。通过使用学生交互数据的机器学习方法，该预测器可预测学生何时可能需要帮助，并主动触发提示，与按需帮助相比，显著提升了后测表现、帮助的恰当性以及训练效率。

ABSTRACT

Determining when and whether to provide personalized support is a well-known challenge called the assistance dilemma. A core problem in solving the assistance dilemma is the need to discover when students are unproductive so that the tutor can intervene. Such a task is particularly challenging for open-ended domains, even those that are well-structured with defined principles and goals. In this paper, we present a set of data-driven methods to classify, predict, and prevent unproductive problem-solving steps in the well-structured open-ended domain of logic. This approach leverages and extends the Hint Factory, a set of methods that leverages prior student solution attempts to build data-driven intelligent tutors. We present a HelpNeed classification, that uses prior student data to determine when students are likely to be unproductive and need help learning optimal problem-solving strategies. We present a controlled study to determine the impact of an Adaptive pedagogical policy that provides proactive hints at the start of each step based on the outcomes of our HelpNeed predictor: productive vs. unproductive. Our results show that the students in the Adaptive condition exhibited better training behaviors, with lower help avoidance, and higher help appropriateness (a higher chance of receiving help when it was likely to be needed), as measured using the HelpNeed classifier, when compared to the Control. Furthermore, the results show that the students who received Adaptive hints based on HelpNeed predictions during training significantly outperform their Control peers on the posttest, with the former producing shorter, more optimal solutions in less time. We conclude with suggestions on how these HelpNeed methods could be applied in other well-structured open-ended domains.

研究动机与目标

• 解决在开放式、结构良好领域中出现的辅助困境，因为学生常常在需要帮助时未能主动寻求帮助。
• 开发一种基于数据驱动的方法，根据先前的学生行为对低效的问题解决步骤进行分类，标记为 'HelpNeed'。
• 设计并评估一种自适应教学策略，当预测到 HelpNeed 时主动提供提示。
• 通过提高帮助的恰当性并减少帮助回避行为，改善学生的学习成效。
• 将该方法推广至其他结构良好、开放式领域，如数学、物理和编程。

提出的方法

• 扩展 Hint Factory 框架，以定义步骤级别的效率，并将步骤分类为低效（HelpNeed）或高效。
• 使用历史学生解题数据训练机器学习模型，以预测下一步是否可能为低效步骤。
• 应用修改后的贝尔曼方程来量化整体解题质量，并为 HelpNeed 分类提供依据。
• 定义两类低效步骤：'机会型'（效率低下但无需帮助）和 '偏离型'（效率低下且需要帮助）。
• 实施一种自适应教学策略，当 HelpNeed 预测器将某一步标记为可能需要帮助时，主动提供提示。
• 采用基于状态和无状态的分类器构建 HelpNeed 预测器，实现在无需为每个问题重新训练模型的情况下跨问题进行泛化。

实验结果

研究问题

• RQ1 在结构良好、开放式逻辑领域中，数据驱动的 HelpNeed 预测器能否准确识别出低效的问题解决步骤？
• RQ2 基于 HelpNeed 预测结果主动提供提示，是否能显著提升学生在后测中的表现，相比按需帮助？
• RQ3 自适应提示策略是否能带来更好的求助行为，例如减少帮助回避和提高帮助的恰当性？
• RQ4 HelpNeed 预测器和自适应策略对解题最优性及完成时间有何影响？
• RQ5 该方法在多大程度上可推广至其他结构良好、开放式领域，如数学或编程？

主要发现

• 与对照组相比，自适应组学生在后测中表现更优，解题时间更短，且解题路径更短、更优。
• 自适应组的帮助恰当性显著更高，意味着他们更可能在预测需要帮助时获得帮助。
• 自适应组的帮助回避行为更低，且被分类为 HelpNeed 的步骤更少，表明其训练行为得到改善。
• HelpNeed 预测器成功利用先前学生交互数据识别出低效步骤，从而实现有效的主动提示推送。
• 自适应策略同时减少了帮助滥用和错过帮助的机会，实现了干预时机的最优平衡。
• 该方法可推广至其他具有明确定义状态和解题评分机制的领域，但需在新领域中进一步验证。