[Paper Review] Text Recognition in the Wild: A Survey

Xiaoxue Chen, Lianwen Jin+3 more|arXiv (Cornell University)|May 7, 2020

Handwritten Text Recognition Techniques245 references48 citations

TL;DR

A comprehensive survey of scene text recognition (STR), detailing fundamental problems, deep-learning based methodologies, datasets, evaluation protocols, and future directions.

ABSTRACT

The history of text can be traced back over thousands of years. Rich and precise semantic information carried by text is important in a wide range of vision-based application scenarios. Therefore, text recognition in natural scenes has been an active research field in computer vision and pattern recognition. In recent years, with the rise and development of deep learning, numerous methods have shown promising in terms of innovation, practicality, and efficiency. This paper aims to (1) summarize the fundamental problems and the state-of-the-art associated with scene text recognition; (2) introduce new insights and ideas; (3) provide a comprehensive review of publicly available resources; (4) point out directions for future work. In summary, this literature review attempts to present the entire picture of the field of scene text recognition. It provides a comprehensive reference for people entering this field, and could be helpful to inspire future research. Related resources are available at our Github repository: https://github.com/HCIILAB/Scene-Text-Recognition.

Motivation & Objective

• Summarize fundamental problems and challenges in scene text recognition (STR).
• Review end-to-end STR systems and their architectural variants.
• Catalog publicly available datasets, benchmarks, and evaluation protocols for STR.
• Highlight recent advances in deep learning for STR and discuss future directions.

Proposed method

• Classify STR approaches into segmentation-based and segmentation-free categories.
• Describe preprocessing, feature representation, sequence modeling, and prediction stages in segmentation-free pipelines.
• Discuss rectification, text enhancement, and background removal as preprocessing aids.
• Summarize the evolution from hand-crafted features to deep-learning based methods and end-to-end systems.
• Provide a resource-oriented review with public datasets, code, and baselines.

Experimental results

Research questions

• RQ1 What are the core problems and stages involved in scene text recognition (localization, verification, detection, segmentation, recognition, and end-to-end systems) and their unique challenges?
• RQ2 How have end-to-end STR systems evolved with deep learning, and what are the strengths/limitations of segmentation-based vs segmentation-free methods?
• RQ3 What datasets, benchmarks, and evaluation protocols are standard for STR, and how do they compare across methods?
• RQ4 What future directions and open problems remain in STR for improved accuracy and efficiency?

Key findings

• Deep learning dramatically improved STR performance by enabling automatic feature learning and end-to-end optimization.
• Segmentation-free methods (encoder-decoder architectures) have become prominent for handling irregular text without per-character segmentation.
• Rectification, TextSR, and background/foreground preprocessing are important preprocessing components for handling distortion and noise, especially in irregular text.
• Diverse datasets and evaluation protocols exist, with ongoing discussions about real-time performance, scalability, and robustness to background complexity.
• The survey consolidates a baseline for fair method comparisons and points to public resources and code repositories for reproducible research.