[Paper Review] Enhanced Learning with Web-Assisted Education

Anna Helga Jónsdóttir, Gunnar Stefánsson|arXiv (Cornell University)|Oct 17, 2013

Statistics Education and Methodologies8 references3 citations

TL;DR

This study evaluates the tutor-web, a free, open-source web-based educational system for statistics and mathematics that uses interactive quizzes to enhance learning. A randomized crossover experiment found no significant difference in exam performance between students using tutor-web and those doing traditional homework, suggesting tutor-web can effectively replace time-intensive written assignments while student feedback indicated strong preference for a blended approach. Item response theory analysis revealed the need for more difficult questions in the item banks to better support diverse learner abilities.

ABSTRACT

An educational system, the tutor-web (http://tutor-web.net), has been developed and used for educational research. The system is accessible and free to use for anyone having access to the Web. It is based on open source software and the teaching material is licensed under the Creative Commons Attribution-ShareAlike License. The system has been used for computer-assisted education in statistics and mathematics. It offers a unique way to structure and link together teaching material and includes interactive quizzes with the primary purpose of increasing learning rather than mere evaluation. The system was used in a course on basic statistics in 2011. Three types of data were gathered during the course. A randomized crossover experiment was conducted to assess the possible difference in learning (measured by repeated exams) between students using the system and students doing regular homework. The difference between the groups was not found to be significant. Responses to quiz questions were collected and analysed with item response theory type models. These analysis were used to improve the item banks. Finally, the students answered an in-class survey regarding their experience using the tutor-web. The responses of the students gave clear indications of student preferences.

Motivation & Objective

• To assess whether web-assisted education via tutor-web improves learning outcomes compared to traditional written homework.
• To evaluate the adequacy of existing quiz item banks using item response theory (IRT) models.
• To understand student perceptions and preferences regarding the tutor-web system for educational use.

Proposed method

• Conducted a randomized crossover experiment comparing exam scores between students using tutor-web quizzes and those completing traditional written homework.
• Applied item response theory (IRT) models to analyze student responses to quiz questions and assess item bank quality.
• Used a probability mass function for adaptive item allocation that shifts difficulty based on student performance, favoring easier items for low-achieving students and harder ones for high-achievers.
• Collected and analyzed student survey data to assess satisfaction, usability, and preferences regarding the tutor-web system.
• Implemented a dynamic item allocation scheme using a difficulty-weighted probability function that adjusts based on student grade and performance history.
• Developed a parametric probability mass function (equation 7) that modulates item selection based on student ability, with a tunable parameter q controlling distribution steepness.

Experimental results

Research questions

• RQ1 Is there a significant difference in learning outcomes between students using tutor-web quizzes and those completing traditional written homework?
• RQ2 Are the existing quiz item banks in tutor-web adequately calibrated to support students of varying ability levels?
• RQ3 What are student perceptions and preferences regarding the use of tutor-web for homework and learning?

Key findings

• No statistically significant difference was found in mean exam scores between students using tutor-web and those doing traditional homework, indicating comparable learning outcomes.
• Students reported high satisfaction with the tutor-web system, but expressed a strong preference for a mixed approach combining both tutor-web and written assignments.
• Item response theory analysis revealed that the current item banks lack sufficient difficult questions, particularly for high-achieving students.
• The probability mass function for item allocation (equation 7) was designed to dynamically adjust question difficulty based on student performance, with higher probabilities assigned to easier items for low-achieving students and harder items for high-achieving students.
• The system's current item allocation method, which used a uniform distribution, was found suboptimal and is being replaced with a performance-based adaptive scheme.
• The study concluded that tutor-web can effectively replace traditional written homework in part, saving time and effort for instructors while maintaining learning outcomes.