[Paper Review] Feature Enhancement Network: A Refined Scene Text Detector

Sheng Zhang, Yuliang Liu+2 more|arXiv (Cornell University)|Nov 12, 2017

Handwritten Text Recognition Techniques27 references17 citations

TL;DR

This paper proposes a Feature Enhancement Network (FEN) for scene text detection, integrating task-specific fusion of low- and high-level features to improve detection of small and thin text. It introduces an adaptively weighted position-sensitive RoI pooling layer and a positives mining strategy to address sample imbalance, achieving state-of-the-art F-measure on ICDAR 2011 and 2013 benchmarks with end-to-end training.

ABSTRACT

In this paper, we propose a refined scene text detector with a extit{novel} Feature Enhancement Network (FEN) for Region Proposal and Text Detection Refinement. Retrospectively, both region proposal with extit{only} $3 imes 3$ sliding-window feature and text detection refinement with extit{single scale} high level feature are insufficient, especially for smaller scene text. Therefore, we design a new FEN network with extit{task-specific}, extit{low} and extit{high} level semantic features fusion to improve the performance of text detection. Besides, since extit{unitary} position-sensitive RoI pooling in general object detection is unreasonable for variable text regions, an extit{adaptively weighted} position-sensitive RoI pooling layer is devised for further enhancing the detecting accuracy. To tackle the extit{sample-imbalance} problem during the refinement stage, we also propose an effective extit{positives mining} strategy for efficiently training our network. Experiments on ICDAR 2011 and 2013 robust text detection benchmarks demonstrate that our method can achieve state-of-the-art results, outperforming all reported methods in terms of F-measure.

Motivation & Objective

• To address the limitations of existing scene text detectors in detecting small and thin text regions.
• To improve detection accuracy by fusing low-level and high-level semantic features in a task-specific manner.
• To resolve sample imbalance during the refinement stage through an effective positives mining strategy.
• To enhance region proposal and detection refinement using an adaptively weighted position-sensitive RoI pooling layer.
• To achieve state-of-the-art performance on standard ICDAR 2011 and 2013 benchmarks for robust scene text detection.

Proposed method

• Proposes a Feature Enhancement Network Stem (FENS) that fuses low-level and high-level features for improved feature representation in text detection.
• Introduces a Feature Enhancement RPN (FE-RPN) and Hyper Feature Generation module to enhance region proposal and detection refinement.
• Designs an adaptively weighted position-sensitive RoI pooling layer to better handle variable aspect ratios in text regions.
• Develops a positives mining strategy to balance foreground and background samples during training, improving model convergence.
• Employs multi-scale testing with non-maximum suppression (NMS) to aggregate predictions across different input resolutions.
• Trains the entire network end-to-end using a combination of region proposal and refinement stages with joint optimization.

Experimental results

Research questions

• RQ1 Can task-specific fusion of low- and high-level features improve detection performance for small and thin scene text?
• RQ2 Does adaptively weighted position-sensitive RoI pooling outperform standard unitary pooling in detecting variable aspect ratio text regions?
• RQ3 Can a dedicated positives mining strategy effectively mitigate sample imbalance during the detection refinement stage?
• RQ4 To what extent does the proposed FEN framework improve F-measure on standard ICDAR benchmarks compared to existing state-of-the-art methods?

Key findings

• The proposed FEN framework achieves a 5% average recall gain and a 3% F-measure gain over the original R-FCN baseline on ICDAR 2011 and 2013 datasets.
• The adaptively weighted position-sensitive RoI pooling improves F-measure by 1.9% on ICDAR 2013 and 1.1% on ICDAR 2011 compared to standard pooling.
• The positives mining strategy improves the foreground-to-background sample ratio from less than 1:4 to approximately 2:5, resolving severe class imbalance.
• With multi-scale testing, the method achieves state-of-the-art F-measure on both ICDAR 2011 and 2013 benchmarks, outperforming all previously reported methods.
• Despite using a shorter input side of 720, the method maintains high performance, indicating robustness to input resolution variations.
• The model maintains competitive inference speed, comparable to other state-of-the-art methods, despite its enhanced architecture.