[Paper Review] DoReMi: First glance at a universal OMR dataset

Elona Shatri, György Fazekas|arXiv (Cornell University)|Jul 16, 2021

Radiomics and Machine Learning in Medical Imaging19 references4 citations

TL;DR

DoReMi introduces a large-scale, universal OMR dataset comprising over 6,400 printed sheet music images with rich, multi-level metadata, including bounding boxes, MIDI, MEI, MusicXML, and PNG files. It achieves 64% mean average precision in object detection using half the data, enabling harmonization with MUSCIMA++ and DeepScores to support end-to-end OMR research and improve evaluation consistency.

ABSTRACT

The main challenges of Optical Music Recognition (OMR) come from the nature of written music, its complexity and the difficulty of finding an appropriate data representation. This paper provides a first look at DoReMi, an OMR dataset that addresses these challenges, and a baseline object detection model to assess its utility. Researchers often approach OMR following a set of small stages, given that existing data often do not satisfy broader research. We examine the possibility of changing this tendency by presenting more metadata. Our approach complements existing research; hence DoReMi allows harmonisation with two existing datasets, DeepScores and MUSCIMA++. DoReMi was generated using a music notation software and includes over 6400 printed sheet music images with accompanying metadata useful in OMR research. Our dataset provides OMR metadata, MIDI, MEI, MusicXML and PNG files, each aiding a different stage of OMR. We obtain 64% mean average precision (mAP) in object detection using half of the data. Further work includes re-iterating through the creation process to satisfy custom OMR models. While we do not assume to have solved the main challenges in OMR, this dataset opens a new course of discussions that would ultimately aid that goal.

Motivation & Objective

• To address the fragmentation in OMR research caused by inconsistent datasets and evaluation protocols.
• To provide a comprehensive, multi-format dataset that supports all stages of OMR, from object detection to semantic encoding.
• To enable harmonization with existing datasets like MUSCIMA++ and DeepScores for improved benchmarking and model comparison.
• To generate high-fidelity, machine-readable metadata (MIDI, MEI, MusicXML) alongside visual data for end-to-end deep learning in OMR.
• To support the development of universal OMR models by offering rich, structured annotations for diverse musical elements.

Proposed method

• The dataset was generated using music notation software to ensure precise alignment between visual elements and semantic metadata.
• Each sheet music image includes bounding boxes for 37 distinct musical symbol classes, such as noteheads, clefs, slurs, and tuplets.
• The dataset provides multiple output formats: PNG images, OMR metadata, MIDI, MEI, and MusicXML for full compatibility with OMR pipelines.
• A baseline object detection model using half the data achieved 64% mean average precision (mAP), validating the dataset's utility.
• The dataset supports both typeset and semantic-level annotations, enabling research on reconstruction and higher-level music understanding.
• The authors designed the dataset to be compatible with MUSCIMA++ and DeepScores, facilitating cross-dataset evaluation and model transfer.

Experimental results

Research questions

• RQ1 Can a unified, multi-format OMR dataset improve consistency and comparability across OMR research?
• RQ2 To what extent can a large-scale, synthetically generated dataset with rich metadata support end-to-end OMR model training?
• RQ3 How well does a baseline object detection model perform on a new, comprehensive OMR dataset like DoReMi?
• RQ4 Can DoReMi be effectively harmonized with existing datasets such as MUSCIMA++ and DeepScores to enable cross-dataset evaluation?
• RQ5 What is the impact of including semantic metadata (e.g., note pitch, duration, dynamics) on OMR model performance and generalization?

Key findings

• The DoReMi dataset contains over 6,400 printed sheet music images with detailed annotations for 37 musical symbol classes.
• The dataset includes multiple output formats—PNG, OMR metadata, MIDI, MEI, and MusicXML—supporting all stages of OMR.
• A baseline object detection model achieved 64% mean average precision (mAP) using only half of the dataset, demonstrating its utility for detection tasks.
• The dataset enables harmonization with MUSCIMA++ and DeepScores, supporting cross-dataset evaluation and model compatibility.
• Semantic metadata such as note pitch, duration, dynamics, and articulations are accurately linked to visual elements, enabling higher-level OMR tasks.
• The use of music notation software ensured high-fidelity alignment between visual and semantic data, reducing annotation errors common in real-world datasets.