[Paper Review] Pyramid Mask Text Detector

Jingchao Liu, Xuebo Liu+4 more|arXiv (Cornell University)|Mar 28, 2019

Handwritten Text Recognition Techniques45 references53 citations

TL;DR

PMTD introduces soft pyramid text masks and a plane clustering method to convert 2D soft masks into accurate 3D pyramid text boxes, achieving state-of-the-art results on ICDAR datasets. It improves over preceding Mask R-CNN based methods by reducing boundary mislabeling and bounding-box dependency.

ABSTRACT

Scene text detection, an essential step of scene text recognition system, is to locate text instances in natural scene images automatically. Some recent attempts benefiting from Mask R-CNN formulate scene text detection task as an instance segmentation problem and achieve remarkable performance. In this paper, we present a new Mask R-CNN based framework named Pyramid Mask Text Detector (PMTD) to handle the scene text detection. Instead of binary text mask generated by the existing Mask R-CNN based methods, our PMTD performs pixel-level regression under the guidance of location-aware supervision, yielding a more informative soft text mask for each text instance. As for the generation of text boxes, PMTD reinterprets the obtained 2D soft mask into 3D space and introduces a novel plane clustering algorithm to derive the optimal text box on the basis of 3D shape. Experiments on standard datasets demonstrate that the proposed PMTD brings consistent and noticeable gain and clearly outperforms state-of-the-art methods. Specifically, it achieves an F-measure of 80.13% on ICDAR 2017 MLT dataset.

Motivation & Objective

• Motivate robust quadrilateral text detection in natural scenes beyond binary masks.
• Address imprecise boundary labeling and error propagation from bounding box predictions in Mask R-CNN based methods.
• Propose soft pyramid labeling to encode shape and location information for text instances.
• Introduce a plane clustering algorithm to regress accurate pyramid (text box) from 3D-encoded mask points.

Proposed method

• Replace binary text masks with soft pyramid labels, where pixel scores are computed by distance-based pyramid geometry within quadrilaterals.
• Compute apex at the text region center with score 1 and boundary as the pyramid base to encode shape and location.
• Modify the mask head with dilated convolutions and bilinear upsampling to enlarge receptive field and reduce checkerboard artifacts.
• Train with pixelwise L1 loss on the soft pyramid mask to reduce boundary mislabeling and improve regression stability.
• During testing, reinterpret the 2D soft mask as 3D points (x,y,z) and apply a plane clustering algorithm to fit four supporting planes, yielding a bottom face whose intersection with z=0 is the output text box.
• Baseline improvements include data augmentation and tailored RPN anchors for text statistics.

Experimental results

Research questions

• RQ1 Can soft pyramid labeling better capture quadrilateral text shapes than binary masks in Mask R-CNN frameworks?
• RQ2 Does plane clustering on 3D-embedded text masks yield more accurate text boxes than direct 2D bounding box estimation or bounding-box-limited masks?
• RQ3 How does PMTD perform on standard benchmarks (ICDAR 2017 MLT, ICDAR 2015, ICDAR 2013) relative to state-of-the-art methods?

Key findings

• PMTD achieves state-of-the-art F-measure on ICDAR 2017 MLT (80.13 with multi-scale testing).
• PMTD shows gains over Baseline Mask R-CNN by about 1.6 percentage points in F-measure on ICDAR 2017 MLT and consistent gains on ICDAR 2015 and ICDAR 2013 datasets.
• Soft pyramid labels reduce boundary mislabeling and improve bounding-box accuracy, especially under higher IoU thresholds.
• Plane clustering yields more accurate text boxes by regressing from a 3D representation of the soft mask, demonstrating robustness to imprecise bounding boxes.
• PMTD outperforms multiple baselines and prior state-of-the-art methods across ICDAR benchmarks, including significant improvements at higher IoU thresholds (e.g., IoU 0.8).
• Ablation studies confirm the contribution of soft labeling and plane clustering to overall performance.