[Paper Review] Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection

Xugong Qin, Yu Zhou+6 more|arXiv (Cornell University)|Sep 8, 2021

Handwritten Text Recognition Techniques71 references4 citations

TL;DR

This paper proposes MAYOR, a reimagined Mask R-CNN framework for dense and arbitrary-shaped scene text detection that addresses two key limitations: learning confusion in the mask head and poor anchor matching for extreme aspect ratio text. By replacing the deconv-conv decoder with an MLP decoder and introducing instance-aware mask learning, MAYOR achieves state-of-the-art performance on five benchmarks, including 88.3% F-measure on DAST1500 and 88.9% on Total-Text with external data.

ABSTRACT

Due to the large success in object detection and instance segmentation, Mask R-CNN attracts great attention and is widely adopted as a strong baseline for arbitrary-shaped scene text detection and spotting. However, two issues remain to be settled. The first is dense text case, which is easy to be neglected but quite practical. There may exist multiple instances in one proposal, which makes it difficult for the mask head to distinguish different instances and degrades the performance. In this work, we argue that the performance degradation results from the learning confusion issue in the mask head. We propose to use an MLP decoder instead of the "deconv-conv" decoder in the mask head, which alleviates the issue and promotes robustness significantly. And we propose instance-aware mask learning in which the mask head learns to predict the shape of the whole instance rather than classify each pixel to text or non-text. With instance-aware mask learning, the mask branch can learn separated and compact masks. The second is that due to large variations in scale and aspect ratio, RPN needs complicated anchor settings, making it hard to maintain and transfer across different datasets. To settle this issue, we propose an adaptive label assignment in which all instances especially those with extreme aspect ratios are guaranteed to be associated with enough anchors. Equipped with these components, the proposed method named MAYOR achieves state-of-the-art performance on five benchmarks including DAST1500, MSRA-TD500, ICDAR2015, CTW1500, and Total-Text.

Motivation & Objective

• To address the performance degradation in Mask R-CNN-based scene text detectors when handling dense text cases where multiple instances fall within a single proposal.
• To resolve the learning confusion issue in the mask head that arises when predicting overlapping or multiple instances within one RoI.
• To improve anchor matching for text instances with extreme aspect ratios by replacing fixed anchor designs with adaptive label assignment.
• To enable robust detection of arbitrary-shaped and curved text by enhancing global instance representation in the mask branch.

Proposed method

• Replacing the standard 'deconv-conv' decoder in the mask head with a learnable MLP decoder to reduce feature confusion and improve mask prediction fidelity.
• Introducing instance-aware mask learning, where the mask head predicts the complete shape of an entire instance rather than classifying each pixel as text or non-text.
• Proposing adaptive label assignment (ALA) in RPN to ensure that text instances with extreme aspect ratios are matched with sufficient anchors, improving training stability and recall.
• Leveraging RoI feature representations to guide the mask head in reconstructing complete, compact, and separated masks for each instance.
• Training the model end-to-end with cross-entropy and dice loss for mask prediction, optimizing for both accuracy and boundary precision.
• Validating the method on five public benchmarks: DAST1500, MSRA-TD500, ICDAR2015, CTW1500, and Total-Text, with ablation studies on COCO for generalization.

Experimental results

Research questions

• RQ1 How does replacing the deconv-conv decoder with an MLP decoder affect mask prediction quality in dense text scenarios?
• RQ2 Can instance-aware mask learning reduce learning confusion and improve mask compactness and separation in the mask head?
• RQ3 To what extent does adaptive label assignment improve detection performance for text instances with extreme aspect ratios?
• RQ4 How does the proposed method perform on arbitrary-shaped and curved text detection compared to existing SOTA methods?
• RQ5 Does the proposed framework generalize beyond scene text detection, as evidenced by performance gains on general instance segmentation benchmarks like COCO?

Key findings

• MAYOR achieves state-of-the-art performance with 88.3% F-measure on DAST1500, outperforming previous SOTA methods by a significant margin.
• On CTW1500, MAYOR achieves 85.3% F-measure without external data and 86.1% with MLT pretraining, surpassing methods like TextSnake and DB.
• On Total-Text, MAYOR reaches 86.3% F-measure without external data and 88.9% with MLT pretraining, outperforming SD by 2% and setting a new SOTA.
• The model improves mask AP on COCO from 35.1% to 35.6% by reducing fragmentary predictions, demonstrating generalization to long, thin instances.
• Ablation studies confirm that both the MLP decoder and instance-aware mask learning contribute significantly to performance gains, especially in dense and curved text scenarios.
• The adaptive label assignment mechanism effectively increases positive sample assignment for long and thin text, improving recall and robustness across datasets.