[Paper Review] LV-ROVER: Lexicon Verified Recognizer Output Voting Error Reduction

Bruno Stuner, Clément Chatelain+1 more|ArXiv.org|Jul 24, 2017

Internet Traffic Analysis and Secure E-voting17 references3 citations

TL;DR

LV-ROVER proposes a low-complexity, lexicon-verified ROVER framework that efficiently combines hundreds of raw BLSTM-based handwritten text line recognizers without requiring language models. By integrating word-level alignment and lexicon-based voting, it achieves state-of-the-art performance on the RIMES dataset with a 2.53% CER and 7.84% WER, outperforming classical ROVER and prior SOTA methods.

ABSTRACT

Offline handwritten text line recognition is a hard task that requires both an efficient optical character recognizer and language model. Handwriting recognition state of the art methods are based on Long Short Term Memory (LSTM) recurrent neural networks (RNN) coupled with the use of linguistic knowledge. Most of the proposed approaches in the literature focus on improving one of the two components and use constraint, dedicated to a database lexicon. However, state of the art performance is achieved by combining multiple optical models, and possibly multiple language models with the Recognizer Output Voting Error Reduction (ROVER) framework. Though handwritten line recognition with ROVER has been implemented by combining only few recognizers because training multiple complete recognizers is hard. In this paper we propose a Lexicon Verified ROVER: LV-ROVER, that has a reduce complexity compare to the original one and that can combine hundreds of recognizers without language models. We achieve state of the art for handwritten line text on the RIMES dataset.

Motivation & Objective

• To address the challenge of combining hundreds of raw handwritten text line recognizers efficiently, especially when language models are unavailable.
• To reduce the computational complexity of traditional ROVER by avoiding dynamic programming-based alignment across multiple recognizers.
• To improve recognition accuracy by introducing a lexicon-verified voting mechanism that enhances word-level consistency and error reduction.
• To enable high-performance recognition using only raw optical models, minimizing dependency on language models or dataset-specific lexicons.
• To demonstrate that large, diverse recognizers can be effectively combined even without linguistic smoothing, achieving SOTA results.

Proposed method

• The LV-ROVER framework introduces a novel alignment module that aligns recognizer outputs based on word count rather than character-level dynamic programming, significantly reducing computational cost.
• It employs a voting mechanism that verifies candidate words against a lexicon, discarding non-lexical outputs and improving recognition reliability.
• The method supports the combination of hundreds of raw BLSTM-based optical models, eliminating the need for language model integration during fusion.
• A training trick is used to generate a diverse cohort of complementary BLSTM recognizers through data augmentation and model weight perturbation.
• The system uses a multi-lexicon strategy, including large general-purpose dictionaries, to maintain high coverage while preserving performance.
• The framework is designed to be lexicon-agnostic, allowing use with any lexicon or even without one, while maintaining robustness and low error rates.

Experimental results

Research questions

• RQ1 Can a ROVER-like framework be designed to efficiently combine hundreds of raw handwritten text line recognizers with minimal computational overhead?
• RQ2 How does lexicon verification improve recognition accuracy when language models are not used during fusion?
• RQ3 To what extent does the performance of LV-ROVER depend on lexicon size or coverage compared to traditional ROVER?
• RQ4 Can a system combining only raw optical models achieve state-of-the-art performance on handwritten text recognition benchmarks?
• RQ5 How does LV-ROVER compare in performance and complexity to classical ROVER and other SOTA methods on the RIMES dataset?

Key findings

• LV-ROVER achieves a character error rate (CER) of 2.53% and word error rate (WER) of 7.84% on the RIMES test set using the Rimes lexicon, outperforming classical ROVER and prior SOTA methods.
• The system maintains stable performance across different lexicon sizes, with CER increasing only slightly (to 3.17%) when using a massive 3.3M-word lexicon without the dataset-specific Rimes lexicon.
• LV-ROVER outperforms state-of-the-art methods such as Voigtlaender et al. (2016) and Pham et al. (2014), achieving a 2.53% CER compared to 2.8% and 3.3% respectively.
• The method shows low sensitivity to lexicon size, indicating robustness and scalability to large or multilingual lexicons.
• Classical ROVER underperforms LV-ROVER when combining raw recognizers (CER 4.73% vs. 2.53%), highlighting the advantage of lexicon verification and efficient alignment.
• The framework enables high-accuracy recognition without language models, demonstrating that raw optical model ensembles can be effectively fused using lexicon-verified voting.