[Paper Review] Open Images V5 Text Annotation and Yet Another Mask Text Spotter

Ilya Krylov, S. K. Nosov+1 more|arXiv (Cornell University)|Jun 23, 2021

Handwritten Text Recognition Techniques18 references8 citations

TL;DR

This paper introduces the largest publicly available human-annotated text dataset for Open Images V5, featuring precise localization and transcription of scene text instances. Using this dataset, the authors train a simple Mask-RCNN-based model, Yet Another Mask Text Spotter (YAMTS), which achieves state-of-the-art or competitive performance on ICDAR 2013, ICDAR 2015, and Total-Text benchmarks, with 74.5% end-to-end recognition on Total-Text after fine-tuning.

ABSTRACT

A large scale human-labeled dataset plays an important role in creating high quality deep learning models. In this paper we present text annotation for Open Images V5 dataset. To our knowledge it is the largest among publicly available manually created text annotations. Having this annotation we trained a simple Mask-RCNN-based network, referred as Yet Another Mask Text Spotter (YAMTS), which achieves competitive performance or even outperforms current state-of-the-art approaches in some cases on ICDAR2013, ICDAR2015 and Total-Text datasets. Code for text spotting model available online at: https://github.com/openvinotoolkit/training_extensions. The model can be exported to OpenVINO-format and run on Intel CPUs.

Motivation & Objective

• To create and release the largest publicly available manually annotated text dataset for Open Images V5, enabling improved training of scene text spotting models.
• To develop a simple yet effective Mask-RCNN-based model (YAMTS) for end-to-end text spotting that performs competitively without complex architectural modifications.
• To demonstrate that training on the newly released Open Images V5 text annotations significantly improves model generalization and performance on standard benchmarks.
• To enable deployment of the trained model on Intel CPUs via OpenVINO™-format export for efficient inference.

Proposed method

• The authors collected and released a large-scale, human-annotated dataset of text instances from Open Images V5, including precise bounding polygons and transcriptions.
• They trained a Mask-RCNN-based architecture with an added text recognition head, using standard training procedures without specialized modules or data augmentation tricks.
• The model was fine-tuned by freezing all layers except the text recognition head, optimizing for 130K iterations to improve recognition accuracy.
• Model inference was enabled via export to OpenVINO™ format, allowing efficient deployment on Intel CPUs.
• Performance was evaluated on ICDAR 2013, ICDAR 2015, Total-Text, and Open Images V5 validation sets using standard metrics: word spotting and end-to-end recognition with various lexicons.
• Ablation studies were conducted by retraining the model without the Open Images V5 text annotations to isolate their contribution to performance gains.

Experimental results

Research questions

• RQ1 How does training on a large-scale, high-quality human-annotated scene text dataset like Open Images V5 impact the performance of a standard Mask-RCNN-based text spotting model?
• RQ2 Can a simple Mask-RCNN-based architecture with a standard text recognition head achieve state-of-the-art performance on standard benchmarks without architectural complexity or specialized modules?
• RQ3 To what extent does fine-tuning only the text recognition head improve recognition accuracy, especially on curved and multi-oriented text?
• RQ4 How does the inclusion of Open Images V5 text annotations affect model generalization, as measured by performance on unseen datasets like Total-Text and ICDAR 2015?

Key findings

• YAMTS achieved 74.1% end-to-end recognition on ICDAR 2015 using a generic lexicon and input resolution of 1280x768, outperforming several prior methods.
• After fine-tuning the text recognition head while freezing other layers, YAMTS reached 74.5% end-to-end recognition on Total-Text, matching the performance of Mask TextSpotter v3 fine-tuned on the larger TextOCR dataset.
• The model achieved 73.7% end-to-end recognition on Total-Text without fine-tuning, demonstrating strong baseline performance on curved and multi-oriented text.
• Ablation experiments showed that excluding Open Images V5 text annotations led to a significant drop in performance, confirming the dataset's critical role in model generalization.
• On the Open Images V5 validation set, YAMTS achieved 56.3% end-to-end recognition with 1600x960 input resolution, indicating strong zero-shot generalization to the source dataset.
• The results demonstrate that high-quality, large-scale human-annotated data can significantly enhance model performance even with a simple, standard architecture.