[Paper Review] Correct Like Humans: Progressive Learning Framework for Chinese Text Error Correction

Yinghui Li, Shirong Ma+6 more|arXiv (Cornell University)|Jun 30, 2023

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes ProTEC, a model-agnostic progressive multi-task learning framework for Chinese Text Error Correction (CTEC) that decomposes the task into three sub-tasks—Error Detection, Error Type Identification, and Correction Result Generation—trained progressively to reduce over-correction. The framework improves correction performance across multiple models with only a 5–10% increase in training and inference time.

ABSTRACT

Chinese Text Error Correction (CTEC) aims to detect and correct errors in the input text, which benefits human daily life and various downstream tasks. Recent approaches mainly employ Pre-trained Language Models (PLMs) to resolve CTEC. Although PLMs have achieved remarkable success in CTEC, we argue that previous studies still overlook the importance of human thinking patterns. To enhance the development of PLMs for CTEC, inspired by humans' daily error-correcting behavior, we propose a novel model-agnostic progressive learning framework, named ProTEC, which guides PLMs-based CTEC models to learn to correct like humans. During the training process, ProTEC guides the model to learn text error correction by incorporating these sub-tasks into a progressive paradigm. During the inference process, the model completes these sub-tasks in turn to generate the correction results. Extensive experiments and detailed analyses demonstrate the effectiveness and efficiency of our proposed model-agnostic ProTEC framework.

Motivation & Objective

• Address the persistent issue of over-correction in Chinese Text Error Correction (CTEC), especially critical in precision-sensitive applications.
• Mitigate error propagation and improve model generalization by structuring the CTEC task into a hierarchy of progressively complex sub-tasks.
• Develop a model-agnostic framework that enhances performance across diverse CTEC models without architectural modification.
• Demonstrate that learning from easy to difficult sub-tasks improves both accuracy and robustness in error correction.
• Validate the effectiveness of progressive learning through ablation studies and hyperparameter analysis on public benchmarks.

Proposed method

• Decompose the CTEC task into three sub-tasks ordered by increasing difficulty: Error Detection (identifying erroneous tokens), Error Type Identification (classifying error types), and Correction Result Generation (producing the final corrected sentence).
• Integrate these sub-tasks into a multi-task learning objective during training, where each sub-task is supervised with ground-truth labels.
• Use the output of earlier sub-tasks (e.g., detected errors and their types) as conditional input for subsequent sub-tasks during inference to constrain correction space.
• Apply learnable loss weights (λ and μ) for Error Detection and Error Type Identification, tuned to balance sub-task contributions during training.
• Ensure model-agnostic compatibility by allowing the framework to be applied to any existing CTEC model, such as MacBERT or BART, without architectural changes.
• During inference, perform sub-tasks sequentially: first detect errors, then classify their types, and finally generate corrections only on erroneous tokens, reducing over-correction risk.

Experimental results

Research questions

• RQ1 Can decomposing CTEC into progressively harder sub-tasks improve correction performance and reduce over-correction?
• RQ2 How does the inclusion of intermediate sub-tasks (Error Detection and Error Type Identification) affect the final correction accuracy?
• RQ3 To what extent does using ground-truth labels from earlier sub-tasks improve the performance of the correction generation stage?
• RQ4 How sensitive is the framework’s performance to the hyper-parameters controlling loss weights for sub-tasks?
• RQ5 Can the proposed framework be effectively applied to diverse CTEC models without architectural modifications?

Key findings

• ProTEC improves F1 scores on the SIGHAN14 dataset by 2.5 percentage points (from 64.7 to 67.2) when applied to MacBERT, demonstrating significant performance gains.
• On the NLPCC dataset, ProTEC boosts the F0.5 score from 39.39 (BART-Chinese) to 42.49, showing consistent improvement across different model architectures.
• Ablation studies confirm that removing either Error Detection or Error Type Identification sub-task leads to a drop in performance, validating their role in reducing error propagation.
• When provided with ground-truth labels for both Error Detection and Error Type Identification, the correction model achieves an F1 of 92.6, indicating that the framework effectively leverages intermediate supervision.
• Hyperparameter analysis shows that setting loss weights λ = μ = 1.0 yields optimal performance, and model performance is relatively insensitive to small variations in these weights.
• The framework increases training and inference time by only 5–10% compared to baselines, confirming its efficiency and practicality for real-world deployment.