[Paper Review] TextMountain: Accurate Scene Text Detection via Instance Segmentation

Yixing Zhu, Jun Du|arXiv (Cornell University)|Nov 30, 2018

Handwritten Text Recognition Techniques33 references18 citations

TL;DR

TextMountain proposes a novel scene text detection method using instance segmentation via text center-border probability (TCBP) and text center-direction (TCD) prediction, enabling accurate separation and grouping of text instances. It achieves state-of-the-art F-measure of 76.85% on MLT and 83.2% on SCUT-CTW1500, with efficient GPU-accelerated post-processing for high-speed inference.

ABSTRACT

In this paper, we propose a novel scene text detection method named TextMountain. The key idea of TextMountain is making full use of border-center information. Different from previous works that treat center-border as a binary classification problem, we predict text center-border probability (TCBP) and text center-direction (TCD). The TCBP is just like a mountain whose top is text center and foot is text border. The mountaintop can separate text instances which cannot be easily achieved using semantic segmentation map and its rising direction can plan a road to top for each pixel on mountain foot at the group stage. The TCD helps TCBP learning better. Our label rules will not lead to the ambiguous problem with the transformation of angle, so the proposed method is robust to multi-oriented text and can also handle well with curved text. In inference stage, each pixel at the mountain foot needs to search the path to the mountaintop and this process can be efficiently completed in parallel, yielding the efficiency of our method compared with others. The experiments on MLT, ICDAR2015, RCTW-17 and SCUT-CTW1500 databases demonstrate that the proposed method achieves better or comparable performance in terms of both accuracy and efficiency. It is worth mentioning our method achieves an F-measure of 76.85% on MLT which outperforms the previous methods by a large margin. Code will be made available.

Motivation & Objective

• Address the challenge of detecting long, multi-oriented, and curved text lines in natural scenes with high accuracy.
• Overcome ambiguity in text instance separation caused by traditional semantic segmentation or binary center-border classification.
• Reduce post-processing time in inference by enabling parallel grouping of pixels via TCBP mountain-like structure.
• Achieve robust performance across diverse text shapes and orientations without relying on fixed polygon vertex ordering.
• Enable efficient, scalable inference suitable for high-resolution images and real-time applications.

Proposed method

• Predict a text center-border probability (TCBP) map where pixel values decay from 1 (text center) to 0 (text border), forming a 'mountain' structure.
• Introduce text center-direction (TCD) to guide TCBP learning and improve feature representation without being used in inference.
• Use the rising direction of TCBP to plan paths from border pixels to the mountaintop, enabling group-wise instance separation.
• Implement a GPU-accelerated parallel grouping algorithm that processes all pixels simultaneously, drastically reducing post-processing time.
• Train a fully convolutional network (FCN) to jointly predict text score (TS), TCBP, and TCD maps end-to-end.
• Apply label rules that are invariant to rotation and curvature, ensuring robustness to multi-oriented and curved text.

Experimental results

Research questions

• RQ1 Can a learned TCBP map with a mountain-like structure effectively separate adjacent text instances without relying on fixed bounding boxes?
• RQ2 Does predicting TCD improve the quality of TCBP learning and lead to better generalization on curved and multi-oriented text?
• RQ3 Can parallel path-finding from border to center in TCBP maps significantly accelerate post-processing compared to sequential grouping?
• RQ4 How does the method perform on datasets with long, multi-oriented, and curved text lines compared to existing SOTA methods?
• RQ5 Is the proposed method robust to image transformations such as rotation and scaling without introducing label ambiguity?

Key findings

• TextMountain achieves an F-measure of 76.85% on the MLT dataset, significantly outperforming previous methods.
• On SCUT-CTW1500, it achieves 82.9% precision, 83.4% recall, and 83.2% F-measure, demonstrating strong performance on curved text detection.
• The method runs at 10.5 FPS on ICDAR2015, with GPU-based grouping being over 200× faster than CPU-based cython implementation.
• On high-resolution RCTW-17 images (long side 1500), GPU grouping takes only 0.0013s per image, compared to 1.2910s with CPU grouping.
• The method maintains high accuracy on long text lines without requiring a receptive field that covers the entire line, due to the TCBP-based grouping mechanism.
• The TCD component improves TCBP learning, as evidenced by consistent performance gains across all benchmark datasets.