[Paper Review] Inkorrect: Online Handwriting Spelling Correction

Andrii Maksai, Henry A. Rowley+2 more|arXiv (Cornell University)|Feb 28, 2022

Handwritten Text Recognition Techniques4 citations

TL;DR

Inkorrect proposes a data- and label-efficient method for online handwriting spelling correction (DISC) that bypasses the need for multiple writer samples or character-level segmentation. By introducing new automated metrics—CDE for similarity and CER/WER for recognizability—it demonstrates superior performance over prior work, achieving a Pareto frontier that dominates existing methods in both similarity and recognizability.

ABSTRACT

We introduce Inkorrect, a data- and label-efficient approach for online handwriting (Digital Ink) spelling correction - DISC. Unlike previous work, the proposed method does not require multiple samples from the same writer, or access to character level segmentation. We show that existing automatic evaluation metrics do not fully capture and are not correlated with the human perception of the quality of the spelling correction, and propose new ones that correlate with human perception. We additionally surface an interesting phenomenon: a trade-off between the similarity and recognizability of the spell-corrected inks. We further create a family of models corresponding to different points on the Pareto frontier between those two axes. We show that Inkorrect's Pareto frontier dominates the points that correspond to prior work.

Motivation & Objective

• Address the lack of scalable, privacy-preserving data collection for digital ink spelling correction (DISC), which traditionally requires multiple samples from the same writer or character segmentation.
• Overcome the limitations of existing evaluation metrics that fail to correlate with human perception of spelling correction quality.
• Introduce a new trade-off framework between similarity to the original ink and recognizability of the corrected word, enabling fine-grained control over correction quality.
• Develop a segmentation-free, unpaired data approach that preserves writer style while generating accurate, recognizable corrections without relying on external segmentation systems.
• Demonstrate that the proposed method outperforms state-of-the-art approaches using both automated metrics and human evaluation, with a Pareto frontier that dominates prior work.

Proposed method

• Propose a novel conditional generative model that conditions on original ink and a target spelling-corrected label, using a style encoder to extract writer-specific features without requiring multiple samples from the same writer.
• Introduce a masking mechanism on the style representation layer to control the trade-off between similarity (CDE) and recognizability (CER/WER), enabling dynamic adjustment of correction fidelity during inference.
• Use a feature-level masking strategy that selectively blocks parts of the style vector, allowing fine-grained control over how much original ink information is preserved versus replaced by the corrected label.
• Train the model on unpaired data—original ink and corrected label pairs without writer IDs or alignment—making it scalable and privacy-preserving.
• Design a new similarity metric, CDE (Character Distance Error), which accounts for the expected differences in ink structure between original and corrected words, improving correlation with human perception.
• Implement a family of models by varying the masking ratio and style dimension, enabling exploration of the similarity-recognizability trade-off and selection of optimal points on the Pareto frontier.

Experimental results

Research questions

• RQ1 Can a data-efficient, segmentation-free approach achieve high-quality digital ink spelling correction without requiring multiple samples from the same writer?
• RQ2 Do existing automatic evaluation metrics such as CER and WER fully capture human perception of correction quality, or is there a need for new metrics?
• RQ3 Is there a measurable trade-off between similarity to the original ink and recognizability of the corrected word in handwriting spelling correction?
• RQ4 Can fine-grained control over the style representation layer enable a controllable trade-off between similarity and recognizability in DISC?
• RQ5 Does the proposed method outperform state-of-the-art approaches in both automated metrics and human evaluation, particularly in balancing similarity and recognizability?

Key findings

• The proposed CDE metric correlates strongly with human perception of similarity, outperforming traditional metrics like CER and WER in capturing correction quality.
• Inkorrect achieves a Pareto frontier that dominates prior work: at similarity level sim=0.8, it outperforms Deepwriting (DW) on all metrics—CDE, CER, and WER—while maintaining better balance between similarity and recognizability.
• With a style dimension of 50, the model achieves both lower CDE (4.5) and lower CER (8.6) than DW, and outperforms models with style dimensions of 10 and 250 in both metrics.
• Feature masking (masking specific dimensions of the style vector) yields better results than timestep masking (skipping parts of the input sequence), with CDE and CER values of 4.5 and 8.6 respectively, compared to 4.7 and 8.0 for random timestep skipping.
• The absence of any masking scheme results in poor generalization, with CDE=4.3 and CER=12.9, indicating that masking is essential for balancing similarity and recognizability.
• Human evaluation confirms that the Inkorrect model with style dimension 50 is preferred over DW in both similarity and recognizability, validating the effectiveness of the new metrics and training approach.