[Paper Review] RRPN++: Guidance Towards More Accurate Scene Text Detection

Jianqi Ma|arXiv (Cornell University)|Sep 28, 2020

Handwritten Text Recognition Techniques54 references4 citations

TL;DR

RRPN++ improves scene text detection by introducing an anchor-free Pyramid Proposal Network (APPN) for efficient, scale-adaptive proposal generation and integrating a recognition branch for multi-task learning and joint filtering. This yields a 6% F-measure gain over RRPN on ICDAR2015, achieving state-of-the-art performance with faster inference speed.

ABSTRACT

RRPN is among the outstanding scene text detection approaches, but the manually-designed anchor and coarse proposal refinement make the performance still far from perfection. In this paper, we propose RRPN++ to exploit the potential of RRPN-based model by several improvements. Based on RRPN, we propose the Anchor-free Pyramid Proposal Networks (APPN) to generate first-stage proposals, which adopts the anchor-free design to reduce proposal number and accelerate the inference speed. In our second stage, both the detection branch and the recognition branch are incorporated to perform multi-task learning. In inference stage, the detection branch outputs the proposal refinement and the recognition branch predicts the transcript of the refined text region. Further, the recognition branch also helps rescore the proposals and eliminate the false positive proposals by the jointing filtering strategy. With these enhancements, we boost the detection results by $6\%$ of F-measure in ICDAR2015 compared to RRPN. Experiments conducted on other benchmarks also illustrate the superior performance and efficiency of our model.

Motivation & Objective

• To address the limitations of RRPN, particularly the reliance on manually designed anchors and coarse proposal refinement.
• To improve detection accuracy and inference speed by replacing anchor-based proposals with an anchor-free, multi-scale proposal network.
• To leverage recognition supervision as a guidance signal to enhance detection performance through multi-task learning.
• To reduce false positives via joint filtering using detection and recognition scores.
• To achieve state-of-the-art performance across multiple benchmarks including ICDAR2013, ICDAR2015, and COCO-Text.

Proposed method

• Proposes Anchor-free Pyramid Proposal Networks (APPN) that generate proposals directly from feature maps without anchor templates, reducing proposal count and accelerating inference.
• Assigns labels for multi-scale text instances to different feature pyramid levels, enabling balanced and efficient multi-scale detection learning.
• Integrates a recognition branch into the detection head to enable end-to-end multi-task learning, where recognition results guide detection refinement.
• Uses RRoI Align to extract more accurate feature representations for proposal refinement, improving localization accuracy.
• Applies joint filtering using both detection scores and recognition confidence to rescore proposals and eliminate false positives.
• Employs a two-stage inference pipeline: first-stage proposals from APPN, followed by refinement and transcript prediction in the detection and recognition branches, respectively.

Experimental results

Research questions

• RQ1 Can an anchor-free proposal generation mechanism improve the efficiency and accuracy of scene text detection compared to anchor-based methods?
• RQ2 How does integrating a recognition branch enhance detection performance through multi-task learning?
• RQ3 Can joint filtering using detection and recognition scores effectively reduce false positive proposals?
• RQ4 To what extent does the proposed APPN design improve multi-scale text detection performance?
• RQ5 Does the integration of recognition supervision lead to measurable gains in F-measure across standard benchmarks?

Key findings

• RRPN++ achieves a 6% absolute improvement in F-measure over RRPN on the ICDAR2015 benchmark, reaching 89.5% F-measure.
• The model achieves the highest F-measure among all regression-based methods on ICDAR2013, with 92.0% F-measure and 13.3 fps inference speed.
• On COCO-Text, RRPN++ achieves 64.6% F-measure, a 1.6% improvement over the next best method, and outperforms most prior approaches in speed.
• The recognition branch significantly improves detection performance, even though it is not optimized for recognition accuracy, demonstrating the value of cross-task supervision.
• Joint filtering using detection and recognition scores effectively reduces false positives, contributing to the overall performance gain.
• The anchor-free APPN design reduces the number of proposals and eliminates the need for IoU-based label assignment, lowering GPU memory usage and improving training efficiency.