[Paper Review] PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML

Jiaquan Ye, Xianbiao Qi+5 more|arXiv (Cornell University)|May 5, 2021

Handwritten Text Recognition Techniques11 references30 citations

TL;DR

The paper presents a four-subtask pipeline (table structure recognition, text line detection, text line recognition, and box assignment) built on MASTER and PSENet to convert table images into HTML, achieving TEDs scores of 96.84% on development data and 96.32% on final evaluation."

ABSTRACT

This paper presents our solution for ICDAR 2021 competition on scientific literature parsing taskB: table recognition to HTML. In our method, we divide the table content recognition task into foursub-tasks: table structure recognition, text line detection, text line recognition, and box assignment.Our table structure recognition algorithm is customized based on MASTER [1], a robust image textrecognition algorithm. PSENet [2] is used to detect each text line in the table image. For text linerecognition, our model is also built on MASTER. Finally, in the box assignment phase, we associatedthe text boxes detected by PSENet with the structure item reconstructed by table structure prediction,and fill the recognized content of the text line into the corresponding item. Our proposed methodachieves a 96.84% TEDS score on 9,115 validation samples in the development phase, and a 96.32%TEDS score on 9,064 samples in the final evaluation phase.

Motivation & Objective

• Motivate the task of reconstructing table images into HTML for scientific literature parsing.
• Propose a four-subtask framework to separate concerns in table-to-HTML conversion.
• Leverage MASTER for both table structure prediction and text line recognition.
• Use PSENet for robust text-line detection and design a box-assignment strategy to fill HTML cells.

Proposed method

• Divide the problem into four sub-tasks: table structure recognition, text line detection, text line recognition, and box assignment.
• Customize MASTER into a table-structure branch and a separate box-regression branch.
• Use PSENet to detect text lines within table images.
• Train a text-line recognizer based on MASTER on a mixed dataset of single-line and multi-line text.
• Apply a three-rule box-assignment strategy (Center Point Rule, IOU Rule, Distance Rule) to map detected text boxes to HTML table cells.
• Evaluate end-to-end HTML generation with the TEDS metric on PubTabNet data.

Experimental results

Research questions

• RQ1 Can a four-subtask pipeline reliably reconstruct table HTML from images with high structural fidelity?
• RQ2 How do structure prediction and text-line recognition interact to maximize HTML fidelity as measured by TEDS?
• RQ3 What impact do matching strategies (Center Point, IOU, Distance) have on final HTML content accuracy?
• RQ4 What is the effect of data augmentation, optimizer choices, and synchronization techniques on end-to-end TEDS performance?

Key findings

• Achieved 96.84% TEDS on the development-phase validation set (9,115 samples).
• Achieved 96.32% TEDS on the final evaluation set (9,064 samples).
• The end-to-end approach emphasizes the importance of accurate table structure prediction for high TEDS scores.
• Using PSENet for text-line detection provides robust line localization within table cells.
• Box-assignment rules (Center Point, IOU, Distance) effectively map text boxes to reconstructed table structure.