[Paper Review] Improving Text Proposals for Scene Images with Fully Convolutional Networks

Dena Bazazian, Raúl Gómez+4 more|arXiv (Cornell University)|Feb 16, 2017

Handwritten Text Recognition Techniques11 references20 citations

TL;DR

This paper proposes a novel method to enhance text proposal quality in scene images by integrating Fully Convolutional Networks (FCNs) to re-rank proposals generated by the Text Proposals algorithm. By leveraging FCN-predicted text probability heatmaps combined with a suppression strategy using a grouping quality score, the approach achieves state-of-the-art recall with significantly fewer proposals—demonstrating superior performance on ICDAR and COCO-Text benchmarks.

ABSTRACT

Text Proposals have emerged as a class-dependent version of object proposals - efficient approaches to reduce the search space of possible text object locations in an image. Combined with strong word classifiers, text proposals currently yield top state of the art results in end-to-end scene text recognition. In this paper we propose an improvement over the original Text Proposals algorithm of Gomez and Karatzas (2016), combining it with Fully Convolutional Networks to improve the ranking of proposals. Results on the ICDAR RRC and the COCO-text datasets show superior performance over current state-of-the-art.

Motivation & Objective

• To improve the ranking of text proposals in scene images to reduce false positives while maintaining high recall.
• To address the limitation of the original Text Proposals method, which generates many false positives despite high recall.
• To leverage the spatially preserved features of Fully Convolutional Networks (FCNs) for more accurate text region scoring.
• To integrate FCN-based text probability scores with the grouping quality score from Text Proposals for improved proposal ranking.
• To evaluate the effectiveness of different ranking and suppression strategies on real and synthetic datasets.

Proposed method

• The method trains a Fully Convolutional Network (FCN) to predict pixel-wise text probability, producing a coarse heatmap of text regions.
• The FCN heatmap is used to re-rank proposals generated by the Text Proposals algorithm based on the mean text probability within each proposal.
• A suppression strategy combines the FCN’s text probability with the original grouping quality score from Text Proposals to refine ranking.
• The suppression threshold is optimized experimentally, with 0.10 yielding the best performance on ICDAR and COCO-Text datasets.
• Different training data sources (COCO-Text, ICDAR, SYNTH) are evaluated to assess generalization, with minimal performance differences observed.
• The final ranking strategy uses a weighted combination of FCN confidence and grouping quality to prioritize proposals that match full text block extents.

Experimental results

Research questions

• RQ1 Can FCN-predicted text probability heatmaps improve the ranking of text proposals generated by the Text Proposals algorithm?
• RQ2 How does combining FCN scores with the grouping quality score affect proposal recall and precision?
• RQ3 What is the optimal suppression threshold for merging FCN confidence and grouping quality in proposal ranking?
• RQ4 Does training the FCN on real or synthetic data yield better generalization for text proposal ranking?
• RQ5 How does the proposed method compare to the baseline Text Proposals and state-of-the-art approaches on standard benchmarks?

Key findings

• The suppression strategy with a threshold of 0.10 achieved the highest detection rate across all datasets and proposal counts.
• On ICDAR-Challenge4, the suppression strategy (SUP_COCO 0.10) reached 85% detection rate at 1000 proposals, outperforming the baseline (77%) and MTP strategies.
• On COCO-Text, the suppression strategy achieved 78% detection rate at 1000 proposals, surpassing the baseline (63%) and MTP strategies.
• The mean text probability ranking strategy (MTP) performed notably worse than the baseline, as it favored small text fragments over complete text blocks.
• Using the FCN model trained on COCO-Text yielded the best results, with minimal performance differences observed when training on ICDAR or SYNTH data.
• Qualitative results confirmed the method’s robustness in detecting diverse text types, including multi-language, handwritten, and deformed text with varying orientations and lengths.