[Paper Review] Optimal Academic Plan Derived from Articulation Agreements: A Preliminary Experiment on Human-Generated and (Hypothetical) Algorithm-Generated Academic Plans

David Van Nguyen, Shayan Doroudi+1 more|arXiv (Cornell University)|Jul 10, 2023

Scheduling and Timetabling SolutionsDecision Sciences3 citations

TL;DR

This study proposes a prototype optimization tool that automatically generates the minimal set of community college courses needed to satisfy transfer requirements across multiple universities using articulation agreements. In a controlled experiment, students using the prototype created more optimal, faster, and more usable academic plans than those using California’s ASSIST system, demonstrating the value of algorithmic assistance in transfer planning.

ABSTRACT

Our preliminary experiment examined a potential pain point with ASSIST, California's database of articulation agreements. That pain point is cross-referencing multiple articulation agreements to manually develop an optimal academic plan. Optimal is defined as the minimal set of community college courses that satisfy all transfer requirements for the multiple universities a student is preparing to apply to. Accordingly, we designed a low-fidelity prototype that lists the minimal set of courses a hypothetical optimization algorithm would output based on selected articulation agreements. 24 students were tasked with creating an optimal academic plan using either ASSIST (which requires manual optimization) or the optimization prototype (which already provides the minimal set of classes). Prototype users had less optimality mistakes, were faster, and provided higher usability ratings compared to ASSIST users. Going forward, future research needs to move beyond our proof of value of a hypothetical optimization algorithm and towards actually implementing an algorithm.

Motivation & Objective

• Address the challenge of manually cross-referencing multiple articulation agreements to create optimal transfer plans.
• Reduce student errors and time spent on academic planning by automating course selection based on transfer requirements.
• Evaluate the usability and effectiveness of a hypothetical optimization algorithm compared to existing manual systems like ASSIST.
• Demonstrate the potential of algorithmic assistance in improving transfer planning outcomes for community college students.

Proposed method

• Developed a low-fidelity prototype that simulates an optimization algorithm to compute the minimal set of community college courses satisfying all selected university transfer requirements.
• Designed the prototype to process articulation agreements from California’s ASSIST database and identify overlapping or required courses across multiple institutions.
• Implemented a heuristic-based approach to minimize course count while ensuring all transfer requirements are met.
• Conducted a controlled experiment with 24 students, assigning them to either use ASSIST (manual method) or the prototype (algorithmic assistance).
• Measured performance using optimality (accuracy of course selection), time-on-task, and usability ratings.
• Used statistical comparison to assess differences in performance and user experience between the two conditions.

Experimental results

Research questions

• RQ1 Does a prototype algorithmic tool reduce optimality mistakes in academic planning compared to manual methods using ASSIST?
• RQ2 Can an algorithmic assistant reduce the time required to develop a transfer-ready course plan?
• RQ3 How does user satisfaction and perceived usability compare between manual planning and algorithmic assistance?
• RQ4 To what extent does algorithmic assistance improve the accuracy of course selection across multiple university transfer requirements?

Key findings

• Prototype users made significantly fewer optimality mistakes in course selection compared to ASSIST users, indicating higher accuracy in identifying required courses.
• Students using the prototype completed their academic plans faster than those using ASSIST, demonstrating improved efficiency.
• Prototype users reported higher usability ratings, suggesting greater perceived ease and satisfaction with the algorithmic assistance.
• The results support the hypothesis that algorithmic optimization can significantly improve the quality and speed of transfer planning for community college students.