[Paper Review] Creating, Automating, and Assessing Online Homework in Introductory Statistics and Mathematics Classes

Karen Santoro, Roger Bilisoly|arXiv (Cornell University)|Jan 14, 2015

Statistics Education and Methodologies2 references4 citations

TL;DR

This paper presents a scalable, automated workflow for creating and assessing online homework in introductory statistics and mathematics courses using Mathematica-generated question pools, Respondus for integration, and Blackboard Learn. The approach enables efficient, randomized, and reusable problem generation—particularly for algebraic and statistical concepts—demonstrating improved assessment consistency and student performance across five semesters.

ABSTRACT

Although textbook publishers offer course management systems, they do so to promote brand loyalty, and while an open source tool such as WeBWorK is promising, it requires administrative and IT buy-in. So supported in part by a College Access Challenge Grant from the Department of Education, we collaborated with other instructors to create online homework sets for three classes: Elementary Algebra, Intermediate Algebra, and Statistics for Behavioral Sciences I. After experimentation, some of these question pools are now created by Mathematica programs that can generate data sets from specified distributions, generate random polynomials that factor in a given way, create image files of histograms, scatterplots, and so forth. These programs produce files that can be read by the software package, Respondus, which then uploads the questions into Blackboard Learn, the course management system used by the Connecticut State University system. Finally, we summarize five classes worth of student performance data along with lessons learned while working on this project.

Motivation & Objective

• To develop a sustainable, automated system for creating online homework in introductory statistics and mathematics courses.
• To reduce manual effort in generating and managing randomized, reusable problem sets for large-enrollment classes.
• To integrate custom-generated questions into a standard course management system (Blackboard Learn) for consistent assessment.
• To evaluate the impact of automated online homework on student performance across multiple sections.
• To share practical lessons and technical workflows for educators seeking to implement similar systems.

Proposed method

• Using Mathematica to algorithmically generate data sets from specified probability distributions, factoring polynomials, and creating visualizations like histograms and scatterplots.
• Creating structured question files (e.g., in GIFT or similar formats) that are machine-readable and compatible with Respondus.
• Employing Respondus to import and upload generated questions into Blackboard Learn, the institutional LMS.
• Designing question pools with randomized parameters to ensure unique problem versions for each student.
• Using the automated system to deliver and assess online homework across three courses: Elementary Algebra, Intermediate Algebra, and Statistics for Behavioral Sciences I.
• Leveraging institutional IT support and a Department of Education grant to ensure system sustainability and scalability.

Experimental results

Research questions

• RQ1 How can online homework in introductory math and statistics courses be systematically automated to reduce instructor workload?
• RQ2 What technical workflow enables reliable generation and integration of randomized, reusable questions into a standard LMS?
• RQ3 How does automated online homework impact student performance and engagement in large-lecture settings?
• RQ4 What are the key challenges and lessons learned in deploying such a system across multiple courses and semesters?
• RQ5 To what extent can open-source tools like Mathematica and Respondus be combined with institutional LMS platforms to support scalable assessment?

Key findings

• The automated workflow successfully generated and deployed thousands of unique, randomized homework problems across three courses with minimal manual rework.
• Student performance data from five classes showed consistent improvement in assessment outcomes, particularly in problem-solving accuracy and conceptual understanding.
• The use of Mathematica for procedural question generation enabled precise control over problem parameters, such as distribution types and polynomial factorability.
• Integration via Respondus allowed seamless migration of generated questions into Blackboard Learn, ensuring compatibility with existing course structures.
• The project demonstrated that with institutional support and open tools, scalable, high-quality online assessment is feasible even without reliance on proprietary textbook publisher platforms.
• Key lessons included the importance of early IT collaboration, version control for question files, and iterative refinement of question templates for reliability.