[Paper Review] Single Shot Text Detector with Regional Attention

Pan He, Weilin Huang+4 more|arXiv (Cornell University)|Sep 1, 2017

Handwritten Text Recognition Techniques31 references58 citations

TL;DR

A single-shot text detector (SSD-based) with a Text Attention Module (TAM) and a Hierarchical Inception Module (HIM) directly outputs word-level bounding boxes and achieves state-of-the-art results on ICDAR 2013/2015 and COCO-Text.

ABSTRACT

We present a novel single-shot text detector that directly outputs word-level bounding boxes in a natural image. We propose an attention mechanism which roughly identifies text regions via an automatically learned attentional map. This substantially suppresses background interference in the convolutional features, which is the key to producing accurate inference of words, particularly at extremely small sizes. This results in a single model that essentially works in a coarse-to-fine manner. It departs from recent FCN- based text detectors which cascade multiple FCN models to achieve an accurate prediction. Furthermore, we develop a hierarchical inception module which efficiently aggregates multi-scale inception features. This enhances local details, and also encodes strong context information, allow- ing the detector to work reliably on multi-scale and multi- orientation text with single-scale images. Our text detector achieves an F-measure of 77% on the ICDAR 2015 bench- mark, advancing the state-of-the-art results in [18, 28]. Demo is available at: http://sstd.whuang.org/.

Motivation & Objective

• Address the challenge of accurate word-level text detection in natural images with varying scale and orientation.
• Eliminate multi-stage bottom-up processing by delivering direct word bounding boxes in a single pass.
• Enhance feature representations for multi-scale and multi-orientation text through specialized modules.
• Introduce a text-specific supervision signal to learn rough text regions via an attention mechanism.
• Improve robustness and speed to enable practical, single-shot text detection.

Proposed method

• Introduce a Text Attention Module (TAM) that learns a pixel-wise text mask and injects text-region attention into Aggregated Inception Features (AIFs).
• Develop a Hierarchical Inception Module (HIM) that aggregates multi-scale inception features and fuses information across layers to form richer AIFs.
• Integrate TAM and HIM into the SSD framework to produce word-level bounding boxes directly in one shot (with simple NMS).
• Train end-to-end with a pixel-wise text mask loss as auxiliary supervision to guide attention learning.
• Predict N word bounding boxes per spatial location using a multi-scale set of default boxes with diverse scales and aspect ratios, including orientation parameters.
• Evaluate on ICDAR 2013, ICDAR 2015, and COCO-Text to demonstrate state-of-the-art performance and efficiency.

Experimental results

Research questions

• RQ1 Can a single-shot SSD-based detector be extended with text-specific modules to directly predict word-level bounding boxes without post-processing?
• RQ2 Do TAM and HIM improve recall and precision for multi-scale and multi-orientation text detection in natural scenes?
• RQ3 How does the proposed method perform on standard benchmarks (ICDAR 2013/2015, COCO-Text) in terms of accuracy and speed?

Key findings

• Achieved state-of-the-art F-measure on ICDAR 2013 (0.87) and ICDAR 2015 (0.77) benchmarks.
• Outperformed competing methods on COCO-Text with an F-score of 0.37, demonstrating good generalization.
• Single-shot detector with TAM and HIM runs at 0.13 seconds per image on 704x704 inputs on a single GPU.
• TAM and HIM independently improve recall and precision, with TAM+HIM providing the best overall F-measure (0.87 on ICDAR 2013).
• The method maintains high word-level accuracy for small, multi-scale, and multi-orientation text without complex post-processing.