[Paper Review] ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network

Yuliang Liu, Hao Chen+4 more|arXiv (Cornell University)|Feb 24, 2020

Handwritten Text Recognition TechniquesComputer Science44 references37 citations

TL;DR

ABCNet introduces a real-time end-to-end framework that represents arbitrarily-shaped text with adaptive Bezier curves, uses BezierAlign for precise feature sampling, and achieves state-of-the-art accuracy with real-time speed on Total-Text and CTW1500.

ABSTRACT

Scene text detection and recognition has received increasing research attention. Existing methods can be roughly categorized into two groups: character-based and segmentation-based. These methods either are costly for character annotation or need to maintain a complex pipeline, which is often not suitable for real-time applications. Here we address the problem by proposing the Adaptive Bezier-Curve Network (ABCNet). Our contributions are three-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve. 2) We design a novel BezierAlign layer for extracting accurate convolution features of a text instance with arbitrary shapes, significantly improving the precision compared with previous methods. 3) Compared with standard bounding box detection, our Bezier curve detection introduces negligible computation overhead, resulting in superiority of our method in both efficiency and accuracy. Experiments on arbitrarily-shaped benchmark datasets, namely Total-Text and CTW1500, demonstrate that ABCNet achieves state-of-the-art accuracy, meanwhile significantly improving the speed. In particular, on Total-Text, our realtime version is over 10 times faster than recent state-of-the-art methods with a competitive recognition accuracy. Code is available at https://tinyurl.com/AdelaiDet

Motivation & Objective

• Motivate efficient end-to-end spotting of arbitrarily-shaped scene text.
• Propose a compact Bezier-curve representation to fit curved text with minimal overhead.
• Introduce BezierAlign for precise feature sampling enabling a lightweight recognition branch.
• Demonstrate real-time performance with competitive or superior accuracy on Total-Text and CTW1500.
• Provide a Bezier-curve synthetic dataset to augment training for irregular text forms.

Proposed method

• Represent curved scene text with cubic Bezier curves via eight control-point regression.
• Generate Bezier curve ground truth from polygon annotations using least squares fitting.
• Introduce BezierAlign, a non-rectangular sampling layer that aligns features along Bezier curve boundaries for robust recognition.
• Use a lightweight recognition branch (CNN + BiLSTM) with CTC loss connected through BezierAlign features.
• Train end-to-end on synthesized and real data, with Bezier curve ground-truth guiding end-to-end RoI feature extraction.

Experimental results

Research questions

• RQ1 Can cubic Bezier curves accurately represent oriented or curved scene text in a single-stage detector?
• RQ2 Does BezierAlign improve end-to-end recognition accuracy for arbitrarily-shaped text without adding significant computation?
• RQ3 What is the trade-off between sampling point density in BezierAlign and end-to-end performance and speed?
• RQ4 How does ABCNet perform in end-to-end spotting on Total-Text and CTW1500 compared to prior methods?

Key findings

• ABCNet achieves state-of-the-art end-to-end text spotting on Total-Text and CTW1500 while supporting real-time inference.
• BezierAlign substantially improves recognition performance over horizontal or quadrilateral sampling while adding negligible overhead.
• Bezier curve detection incurs almost no extra computation compared to standard bounding-box detection.
• On Total-Text, a real-time single-scale version is over 10x faster than recent SOTA methods with competitive accuracy.
• ABCNet variants (F, MS) show favorable speed-accuracy trade-offs, with BezierAlign driving notable end-to-end gains.