[Paper Review] Calamari - A High-Performance Tensorflow-based Deep Learning Package for Optical Character Recognition

Christoph Wick, Christian Reul+1 more|arXiv (Cornell University)|Jul 5, 2018

Handwritten Text Recognition Techniques24 citations

TL;DR

Calamari is a high-performance, TensorFlow-based deep learning framework for optical character recognition (OCR) that leverages customizable CNN-LSTM architectures trained via Connectionist Temporal Classification (CTC) and native support for pretraining and voting. It achieves state-of-the-art performance with a 0.11% Character Error Rate (CER) on modern English (UW3) and 0.18% on German Fraktur (DTA19), outperforming OCRopy, OCRopus3, and Tesseract 4.

ABSTRACT

Optical Character Recognition (OCR) on contemporary and historical data is still in the focus of many researchers. Especially historical prints require book specific trained OCR models to achieve applicable results (Springmann and L\\"udeling, 2016, Reul et al., 2017a). To reduce the human effort for manually annotating ground truth (GT) various techniques such as voting and pretraining have shown to be very efficient (Reul et al., 2018a, Reul et al., 2018b). Calamari is a new open source OCR line recognition software that both uses state-of-the art Deep Neural Networks (DNNs) implemented in Tensorflow and giving native support for techniques such as pretraining and voting. The customizable network architectures constructed of Convolutional Neural Networks (CNNS) and Long-ShortTerm-Memory (LSTM) layers are trained by the so-called Connectionist Temporal Classification (CTC) algorithm of Graves et al. (2006). Optional usage of a GPU drastically reduces the computation times for both training and prediction. We use two different datasets to compare the performance of Calamari to OCRopy, OCRopus3, and Tesseract 4. Calamari reaches a Character Error Rate (CER) of 0.11% on the UW3 dataset written in modern English and 0.18% on the DTA19 dataset written in German Fraktur, which considerably outperforms the results of the existing softwares.

Motivation & Objective

• To reduce manual annotation effort in training OCR models for historical and contemporary texts.
• To develop a high-performance, open-source OCR system that supports advanced deep learning techniques like pretraining and ensemble voting.
• To improve OCR accuracy on challenging historical scripts, such as German Fraktur, using state-of-the-art deep neural networks.
• To provide a flexible, customizable framework for line-level OCR using CNN-LSTM architectures with CTC loss.
• To enable efficient training and inference through GPU acceleration and optimized TensorFlow implementation.

Proposed method

• The system uses a customizable deep neural network architecture combining Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) layers.
• Training is performed using the Connectionist Temporal Classification (CTC) algorithm to handle sequence-to-sequence alignment without requiring explicit alignment of input and output.
• The framework natively supports pretraining on large-scale unlabeled text data to improve generalization and reduce annotation needs.
• It integrates voting mechanisms across multiple models to enhance prediction robustness and accuracy.
• GPU acceleration is supported to significantly reduce training and inference times.
• The model is trained and evaluated on two benchmark datasets: UW3 (modern English) and DTA19 (German Fraktur).

Experimental results

Research questions

• RQ1 Can a deep learning-based OCR system with native support for pretraining and voting achieve superior performance on historical and contemporary text recognition?
• RQ2 How does the integration of CNN-LSTM architectures with CTC training improve character error rates on challenging scripts like German Fraktur?
• RQ3 To what extent does pretraining reduce the need for large-scale manually annotated ground truth data in OCR?
• RQ4 How does Calamari compare in performance and efficiency to existing OCR tools like Tesseract 4, OCRopus3, and OCRopy?
• RQ5 Can a TensorFlow-based framework with GPU support significantly reduce training and inference times in OCR tasks?

Key findings

• Calamari achieves a Character Error Rate (CER) of 0.11% on the UW3 dataset, which contains modern English text, outperforming OCRopy, OCRopus3, and Tesseract 4.
• On the DTA19 dataset, which contains German Fraktur script, Calamari achieves a CER of 0.18%, demonstrating superior performance on historical scripts.
• The integration of pretraining and voting mechanisms significantly improves model robustness and reduces error rates, especially in low-resource scenarios.
• GPU acceleration enables substantial reductions in training and inference time, enhancing the practicality of training complex models.
• The framework's customizable CNN-LSTM architecture with CTC training provides a strong foundation for high-accuracy line-level OCR.
• Calamari is open-source and designed for extensibility, supporting researchers in adapting the system for diverse OCR applications.