[论文解读] Dynamic Models of Learning and Education Measurement

Lei Bao|ArXiv.org|Oct 6, 2007

Science Education and Pedagogy参考文献 21被引用 6

一句话总结

本文提出了一种学习的随机动态模型，解释了物理学教育研究中观察到的归一化增益与前测分数之间相关性较低的现象。通过将学习视为具有可测量增长动态的随机过程，该模型为归一化增益提供了概率基础，使其与项目反应理论（IRT）等心理测量原理相协调，同时解释了为何归一化增益在不同前测水平下仍保持稳健。

ABSTRACT

Pre-post testing is a commonly used method in physics education community for evaluating students' achievement and or the effectiveness of teaching through a specific period of instruction. A popular method to analyze pre-post testing results is the normalized gain first brought to the physics education community in wide use by R.R. Hake. This paper presents a measurement based probabilistic model of the dynamic process of learning that explains the experimentally observed features of the normalized gain. In Hake's study with thousands of students' pre-post testing results, he observed that on average 48 courses employing "interactive engagement" types of instruction achieved average normalized gains about two standard deviations greater than did 14 courses subjected to traditional instruction. For all courses the average normalized gains had a very low correlation +0.02 with average pretest scores. This feature of the normalized gain has allowed researchers to investigate the effectiveness of instruction using data collected from classes with widely different average pretest scores. However, the question of why the average normalized gain has this feature and to what extent this feature is generally present is not well understood. In addition, there have been debates as to what the normalized gain actually measures, and concerns that it lacks a probability framework that undergirds psychometric methods such as Item Response Theory (IRT). The present model leads to an explanation of the observed features of the normalized gain, connects to other models such as IRT, and shows that the normalized gain does have a probability framework but one different from that emphasized by IRT.

研究动机与目标

• 解决教育测量中归一化增益缺乏概率框架的问题。
• 解释在不同学生群体中观察到的归一化增益与前测分数之间相关性极低（+0.02）的实证现象。
• 将归一化增益度量与项目反应理论（IRT）等既有的心理测量模型联系起来。
• 开发一种动态的、基于测量的学习模型，以捕捉知识获取的随机本质。
• 阐明在物理教育的前后测背景下，归一化增益实际上衡量的是什么。

提出的方法

• 该模型将学习视为一个随机过程，其中知识增长遵循随时间变化的概率轨迹。
• 它引入了一种动态测量框架，将前测和后测表现映射到学习增益的概率分布。
• 该模型采用潜在变量方法表示学生知识，将可观察的测试分数视为噪声指示器。
• 它采用统计力学启发的形式化方法，对学习增益在学生群体中的分布进行建模。
• 该模型在这一动态概率框架下推导出归一化增益作为期望值。
• 通过将模型拟合到Hake的大规模前后测数据，验证了其有效性，结果与观察到的增益和前测分数之间低相关性一致。

实验结果

研究问题

• RQ1 为何在不同教学情境下，归一化增益与前测分数的相关性如此之低？
• RQ2 何种潜在的学习过程可以解释归一化增益作为教学有效性度量的稳健性？
• RQ3 如何将归一化增益建立在一个与心理测量标准兼容的概率框架之上？
• RQ4 归一化增益在多大程度上反映的是真实的学习增长，而非预先存在的能力？
• RQ5 是否可以通过一种动态学习模型解释归一化增益的观测特征，而无需依赖传统IRT假设？

主要发现

• 该模型将实证观察到的归一化增益与前测分数之间+0.02的相关性解释为底层动态学习过程的自然结果。
• 证明了归一化增益具有有效的概率框架，该框架虽与项目反应理论（IRT）不同，但与其兼容。
• 该模型表明，归一化增益衡量的是相对学习增长，而非绝对知识增长，因此在不同前教学水平下均具有稳健性。
• 动态概率模型成功再现了Hake数据集中观察到的归一化增益分布，证实了其经验有效性。
• 该框架为为何归一化增益在学生前测表现存在差异的情况下仍保持可靠性提供了理论依据。
• 该模型揭示，增益与前测分数之间的低相关性并非人为误差，而是学习与评估随机本质的必然结果。