[Paper Review] Focus Is What You Need For Chinese Grammatical Error Correction

Jingheng Ye, Yinghui Li+4 more|arXiv (Cornell University)|Oct 23, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes OneTarget, a training strategy that improves Chinese Grammatical Error Correction (CGEC) by selecting a single reference per source sentence, reducing confusion from multiple valid corrections. Despite fewer training samples, models trained with OneTarget achieve higher precision and state-of-the-art performance on MuCGEC, demonstrating that focused training outperforms multi-reference training in modern NLP models.

ABSTRACT

Chinese Grammatical Error Correction (CGEC) aims to automatically detect and correct grammatical errors contained in Chinese text. In the long term, researchers regard CGEC as a task with a certain degree of uncertainty, that is, an ungrammatical sentence may often have multiple references. However, we argue that even though this is a very reasonable hypothesis, it is too harsh for the intelligence of the mainstream models in this era. In this paper, we first discover that multiple references do not actually bring positive gains to model training. On the contrary, it is beneficial to the CGEC model if the model can pay attention to small but essential data during the training process. Furthermore, we propose a simple yet effective training strategy called OneTarget to improve the focus ability of the CGEC models and thus improve the CGEC performance. Extensive experiments and detailed analyses demonstrate the correctness of our discovery and the effectiveness of our proposed method.

Motivation & Objective

• To investigate the impact of multi-reference training on CGEC model performance.
• To address the confusion caused by multiple equally valid corrections in training data.
• To improve model focus and performance by reducing training data complexity.
• To demonstrate that fewer, refined references lead to better generalization than larger, ambiguous multi-reference sets.

Proposed method

• OneTarget selects one reference per source sentence using strategies like random choice or Levenshtein ratio ranking.
• The method is model-agnostic and applicable to both Seq2Edit-based and Seq2Seq-based CGEC models.
• It constructs a smaller, more coherent training dataset by filtering out redundant references.
• The approach reduces training sample size by 40-50% while maintaining or improving performance.
• Experiments compare models trained on full multi-reference data versus OneTarget-filtered data.
• Ablation studies evaluate performance across datasets with 1 to 3 references per source sentence.

Experimental results

Research questions

• RQ1 Does multi-reference training genuinely improve CGEC model performance, or does it introduce confusion?
• RQ2 Can training with a single, optimal reference per sentence improve model focus and performance?
• RQ3 How does the number of target references per source sentence affect precision and recall in CGEC models?
• RQ4 Is there a trade-off between recall and precision when using multi-reference data?
• RQ5 Can a simpler, focused training strategy outperform larger, more complex multi-reference datasets?

Key findings

• OneTarget-trained models achieve higher F0.5 scores than multi-reference baselines, even with 40-50% fewer training samples.
• The 1-target training setting achieves the highest F0.5 score (31.45) on MuCGEC-dev, outperforming 2-target (31.11) and 3-target (30.92) settings.
• OneTarget improves precision significantly without a major drop in recall, which is critical for grammar correction tasks.
• The method achieved first place in the CCL2022-CLTC competition on MuCGEC-test, confirming its real-world effectiveness.
• Models trained with OneTarget outperform full-dataset baselines in both efficiency and accuracy, especially in precision-oriented settings.
• The study reveals that more training samples do not guarantee better performance if they introduce ambiguity through multiple references.