[Paper Review] TextBoxes: A Fast Text Detector with a Single Deep Neural Network

Minghui Liao, Baoguang Shi+3 more|arXiv (Cornell University)|Nov 21, 2016

Handwritten Text Recognition TechniquesComputer Science26 references445 citations

TL;DR

TextBoxes presents an end-to-end fully convolutional network for fast scene text detection that predicts word bounding boxes in a single forward pass, with high accuracy and speed, and further boosts performance when paired with a recognizer (CRNN) for word spotting and end-to-end recognition.

ABSTRACT

This paper presents an end-to-end trainable fast scene text detector, named TextBoxes, which detects scene text with both high accuracy and efficiency in a single network forward pass, involving no post-process except for a standard non-maximum suppression. TextBoxes outperforms competing methods in terms of text localization accuracy and is much faster, taking only 0.09s per image in a fast implementation. Furthermore, combined with a text recognizer, TextBoxes significantly outperforms state-of-the-art approaches on word spotting and end-to-end text recognition tasks.

Motivation & Objective

• Motivate robust, real-time scene text detection in natural images.
• Develop an end-to-end trainable network that directly predicts word bounding boxes.
• Address large variation in word aspect ratios with specialized output layers and default boxes.
• Demonstrate how combining detection with recognition improves word spotting and end-to-end recognition.

Proposed method

• Use a 28-layer fully convolutional network based on VGG-16 with added text-box layers after multiple conv layers.
• Predict, at each map location, text presence scores and bounding-box offsets to multiple default boxes (including long aspect ratios).
• Employ irregular 1x5 convolutional filters in text-box layers to create rectangular receptive fields suited for word shapes.
• Match default boxes to ground-truth word boxes and optimize a multi-task loss combining classification (softmax) and localization (smooth L1).
• Apply multi-scale input testing (five scales) to further boost accuracy, with standard non-maximum suppression to aggregate outputs.

Experimental results

Research questions

• RQ1 Can a single fully convolutional network directly predict word bounding boxes with high accuracy and speed in natural scenes?
• RQ2 Do irregular inception-style output layers and long-aspect-ratio default boxes improve detection of words with extreme aspect ratios?
• RQ3 Does integrating a text recognizer (CRNN) with detection improve word spotting and end-to-end text recognition performance?
• RQ4 What is the impact of multi-scale inputs on detection accuracy and speed for scene text?

Key findings

• TextBoxes achieves state-of-the-art text localization performance on ICDAR 2011 and ICDAR 2013 benchmarks at high speed.
• On single-scale input, TextBoxes reaches 0.09s per image, and 0.73s per image with multi-scale inputs, on a Titan X GPU.
• Using CRNN for recognition improves word spotting and end-to-end recognition results, providing semantic-level regularization to detection.
• TextBoxes outperforms SSD and other baselines particularly for words with large aspect ratios thanks to the text-box layers and irregular receptive fields.
• When combined with a lexicon, TextBoxes achieves strong end-to-end recognition performance, with recall/precision improvements and competitive F-measures across datasets.