[Paper Review] Educational Timetabling: Problems, Benchmarks, and State-of-the-Art Results

Sara Ceschia, Luca Di Gaspero+1 more|arXiv (Cornell University)|Jan 19, 2022

Scheduling and Timetabling Solutions107 references64 citations

TL;DR

This paper provides a comprehensive survey of standard educational timetabling formulations and benchmarks, identifying six key problem formulations—HTT, CTT, ETT, and their enriched variants—used in major competitions. It reviews state-of-the-art results, highlights reproducibility challenges, and introduces OPTHUB, a web platform for validating and sharing instances, solutions, and code to improve research transparency and comparability in the field.

ABSTRACT

We propose a survey of the research contributions on the field of Educational Timetabling with a specific focus on "standard" formulations and the corresponding benchmark instances. We identify six of such formulations and we discuss their features, pointing out their relevance and usability. Other available formulations and datasets are also reviewed and briefly discussed. Subsequently, we report the main state-of-the-art results on the selected benchmarks, in terms of solution quality (upper and lower bounds), search techniques, running times, statistical distributions, and other side settings.

Motivation & Objective

• To identify and analyze six widely adopted educational timetabling problem formulations that have become de facto standards in the research community.
• To evaluate the usability, relevance, and challenge level of publicly available benchmark instances across these formulations.
• To address reproducibility issues in timetabling research by advocating for standardized data sharing, solution validation, and long-term repository maintenance.
• To propose OPTHUB as a centralized, validated platform for hosting instances, solutions, and software to support fair comparison and future benchmarking.
• To guide researchers in selecting appropriate benchmarks for method evaluation, especially for 'horse race' papers comparing new algorithms.

Proposed method

• Systematically reviews six standard timetabling formulations—HTT, CTT, ETT, and their enriched variants—based on their emergence in international competitions and adoption in the literature.
• Analyzes state-of-the-art results on these benchmarks, including solution quality (upper and lower bounds), search techniques, running times, and statistical distributions.
• Evaluates the practical usability of existing datasets, identifying issues such as outdated repositories, poor documentation, and lack of solution availability.
• Introduces OPTHUB, a web application that enables users to upload, validate, and inspect instances and solutions with timestamped, version-controlled access.
• Proposes a framework for future research infrastructure, advocating for community-wide coordination in maintaining robust, standardized benchmarks and validation tools.
• Stresses the importance of open-source code and reproducible experiments, with OPTHUB aiming to support statistical analysis and fair comparison of algorithms.

Experimental results

Research questions

• RQ1 Which educational timetabling problem formulations have achieved de facto standard status in the research community, and why?
• RQ2 How do the difficulty levels of benchmark instances vary across formulations, and which remain challenging despite advances in solution techniques?
• RQ3 What are the main barriers to reproducibility in timetabling research, and how can they be mitigated through standardized data sharing and validation?
• RQ4 To what extent do current benchmarks support statistically sound algorithm tuning versus final validation?
• RQ5 How can a centralized, community-maintained platform like OPTHUB improve transparency, trust, and progress in timetabling research?

Key findings

• The six standard formulations—HTT, CTT, ETT, and their enriched variants—originated primarily from international timetabling competitions and are now widely used as benchmarks.
• Some benchmark instances, particularly from early competitions, are now too easy to solve optimally, while others remain highly challenging even after 20+ years.
• A significant number of published papers do not provide publicly available datasets, solutions, or source code, undermining reproducibility and fair comparison.
• OPTHUB has been developed as a centralized, validated platform hosting four of the six standard formulations, with support for instance and solution validation, scoreboards, and future software execution.
• The survey identifies a critical need for new, high-quality benchmarks to replace those that are no longer challenging, as well as for better instance generators to support statistical tuning of algorithms.
• The authors advocate for coordinated, community-level efforts to maintain robust research infrastructure, with future competitions serving as key drivers for standardization and adoption.