[Paper Review] TextProposals: a Text-specific Selective Search Algorithm for Word Spotting in the Wild

Lluís Gómez, Dìmosthenis Karatzas|arXiv (Cornell University)|Apr 10, 2016

Handwritten Text Recognition Techniques57 references18 citations

TL;DR

This paper proposes TextProposals, a text-specific object proposals algorithm that generates high-quality word hypotheses by hierarchically grouping connected components using multiple similarity cues, without assuming initial regions correspond to individual characters. It achieves state-of-the-art performance in end-to-end word spotting, surpassing previous methods by over 10% F-score on the challenging ICDAR2015 Incidental Text dataset when combined with a holistic recognizer.

ABSTRACT

Motivated by the success of powerful while expensive techniques to recognize words in a holistic way, object proposals techniques emerge as an alternative to the traditional text detectors. In this paper we introduce a novel object proposals method that is specifically designed for text. We rely on a similarity based region grouping algorithm that generates a hierarchy of word hypotheses. Over the nodes of this hierarchy it is possible to apply a holistic word recognition method in an efficient way. Our experiments demonstrate that the presented method is superior in its ability of producing good quality word proposals when compared with class-independent algorithms. We show impressive recall rates with a few thousand proposals in different standard benchmarks, including focused or incidental text datasets, and multi-language scenarios. Moreover, the combination of our object proposals with existing whole-word recognizers shows competitive performance in end-to-end word spotting, and, in some benchmarks, outperforms previously published results. Concretely, in the challenging ICDAR2015 Incidental Text dataset, we overcome in more than 10 percent f-score the best-performing method in the last ICDAR Robust Reading Competition. Source code of the complete end-to-end system is available at https://github.com/lluisgomez/TextProposals

Motivation & Objective

• To address the limitations of traditional scene text recognition methods that rely on individual character segmentation, which often fail on degraded, cursive, or low-contrast text.
• To develop a method that generates high-quality word proposals without explicit character segmentation, enabling efficient holistic word recognition.
• To improve recall and performance in end-to-end word spotting, especially on challenging incidental text datasets with low-quality or small-scale text.
• To demonstrate that text-specific object proposals outperform generic object proposal algorithms in text detection and recognition tasks.

Proposed method

• The method uses a hierarchical clustering approach that groups connected components based on spatial proximity and visual similarity, treating all components as potential text parts regardless of their origin.
• It constructs multiple similarity hierarchies using complementary cues (e.g., spatial, color, texture) to increase the likelihood that each word instance is captured in at least one hierarchy.
• A novel ranking strategy and non-maximal suppression procedure exploit the inclusion relationships among nodes in the hierarchy to efficiently select top proposals.
• The algorithm does not assume a fixed hierarchy structure (e.g., character → bi-gram → word), allowing flexibility in grouping diverse text forms such as cursive or fragmented text.
• The final word proposals are fed into a holistic word recognizer (e.g., DictNet), enabling end-to-end word spotting with high efficiency and accuracy.
• The system is designed to be robust across multi-language scenarios and diverse image conditions, including low resolution, blur, and complex backgrounds.

Experimental results

Research questions

• RQ1 Can a text-specific object proposal method outperform generic object proposal algorithms in word spotting tasks?
• RQ2 Does a hierarchical grouping strategy based on multiple similarity cues improve recall and quality of word proposals in unconstrained scenes?
• RQ3 Can a method that avoids explicit character segmentation achieve superior performance on challenging incidental text datasets?
• RQ4 How does the integration of text-specific proposals with holistic word recognizers compare to existing end-to-end word spotting pipelines?
• RQ5 To what extent does the method generalize across focused text, incidental text, and multi-language scenarios?

Key findings

• TextProposals achieves a 56.00% F-score on the ICDAR2015 Incidental Text dataset, surpassing the previous best method by over 10 percentage points.
• On the ICDAR2013 Focused Text dataset, the method reaches a 70.71% F-score in end-to-end word spotting, outperforming all prior competition entries.
• With only a few thousand proposals, the method attains high recall across diverse benchmarks, including multi-language and low-quality text scenarios.
• The method demonstrates superior performance on incidental text, where traditional detectors often fail due to small size, low resolution, or non-horizontal orientation.
• The integration of TextProposals with the DictNet holistic recognizer achieves state-of-the-art results, even though the dictionary covers only 70% of the words in the ICDAR2015 test set.
• The proposed ranking and suppression strategy effectively leverages the hierarchical structure, enabling efficient and accurate proposal selection without sacrificing recall.