[Paper Review] MOST: A Multi-Oriented Scene Text Detector with Localization Refinement

Minghang He, Minghui Liao+7 more|arXiv (Cornell University)|Apr 2, 2021

Handwritten Text Recognition Techniques43 references4 citations

TL;DR

This paper proposes MOST, a multi-oriented scene text detector that improves localization accuracy for long and multi-scale text instances through three key innovations: a Text Feature Alignment Module (TFAM) for dynamic receptive field adaptation, Position-Aware Non-Maximum Suppression (PA-NMS) for refining detection quality, and an Instance-wise IoU loss for balanced training. The method achieves state-of-the-art performance on MLT17, MTWI, and MSRA-TD500 benchmarks with a fast inference speed of 51.8 FPS.

ABSTRACT

Over the past few years, the field of scene text detection has progressed rapidly that modern text detectors are able to hunt text in various challenging scenarios. However, they might still fall short when handling text instances of extreme aspect ratios and varying scales. To tackle such difficulties, we propose in this paper a new algorithm for scene text detection, which puts forward a set of strategies to significantly improve the quality of text localization. Specifically, a Text Feature Alignment Module (TFAM) is proposed to dynamically adjust the receptive fields of features based on initial raw detections; a Position-Aware Non-Maximum Suppression (PA-NMS) module is devised to selectively concentrate on reliable raw detections and exclude unreliable ones; besides, we propose an Instance-wise IoU loss for balanced training to deal with text instances of different scales. An extensive ablation study demonstrates the effectiveness and superiority of the proposed strategies. The resulting text detection system, which integrates the proposed strategies with a leading scene text detector EAST, achieves state-of-the-art or competitive performance on various standard benchmarks for text detection while keeping a fast running speed.

Motivation & Objective

• To address the poor localization accuracy of existing detectors on long text instances with extreme aspect ratios.
• To improve detection performance on text instances with large scale variations.
• To enhance the robustness and generalization of one-stage text detectors on diverse datasets, including multi-lingual and web-based images.
• To maintain high inference speed while improving detection quality through lightweight, modular components.

Proposed method

• The Text Feature Alignment Module (TFAM) dynamically adjusts the receptive field of feature maps based on coarse detection predictions, enabling better context modeling for long text.
• Position-Aware Non-Maximum Suppression (PA-NMS) selectively merges raw detections by focusing on reliable predictions based on their spatial positions, reducing geometric bias.
• An Instance-wise IoU loss is introduced to balance the contribution of each text instance in the loss function, ensuring fair optimization across varying scales.
• The method is integrated with the EAST detector, using a ResNet-50-FPN backbone, with a classification head, localization branch, and position-sensitive map prediction head.
• The localization branch consists of a coarse localization head, TFAM, and a refined localization head to improve bounding box regression.
• The overall pipeline remains simple and efficient, enabling fast inference while maintaining high accuracy.

Experimental results

Research questions

• RQ1 Can dynamic receptive field adaptation improve localization accuracy for long text instances with extreme aspect ratios?
• RQ2 Can position-aware suppression reduce geometric bias in NMS by prioritizing reliable detections?
• RQ3 Does instance-wise IoU loss lead to better generalization across text instances of varying scales?
• RQ4 Can the proposed components enhance detection performance on multi-lingual and long text datasets without sacrificing inference speed?

Key findings

• MOST achieves a 90.4% precision, 82.7% recall, and 86.4% F-measure on the MSRA-TD500 test set, outperforming previous SOTA methods by 1.5% in F-measure.
• On the MLT17 benchmark, MOST achieves a 76.7% F-measure, surpassing all compared methods, including two-stage models, with a 1.9% higher recall rate.
• On the MTWI dataset, MOST achieves a 74.7% F-measure, outperforming all baselines by at least 1.2%, demonstrating strong generalization to multi-lingual and web-based text.
• The method achieves a fast inference speed of 51.8 FPS on the MLT17 validation set, maintaining high accuracy while keeping a simple pipeline.
• Ablation studies confirm that each component—TFAM, PA-NMS, and Instance-wise IoU loss—contributes significantly to performance gains, with a 4.0% and 9.5% improvement under different IoU thresholds.
• The proposed strategies are generalizable and can be readily extended to other one-stage text detection frameworks.