[Paper Review] Incorporating Terminology Constraints in Automatic Post-Editing

David Wan, Chris Kedzie+3 more|arXiv (Cornell University)|Oct 19, 2020

Natural Language Processing Techniques13 references4 citations

TL;DR

This paper introduces terminology-constrained automatic post-editing (APE) using autoregressive and non-autoregressive Transformer models to preserve domain-specific lexical terms in machine translation outputs. It demonstrates that constrained APE maintains 95% terminology preservation while improving translation quality, and proposes data augmentation to enhance robustness against rare or semantically divergent constraints.

ABSTRACT

Users of machine translation (MT) may want to ensure the use of specific lexical terminologies. While there exist techniques for incorporating terminology constraints during inference for MT, current APE approaches cannot ensure that they will appear in the final translation. In this paper, we present both autoregressive and non-autoregressive models for lexically constrained APE, demonstrating that our approach enables preservation of 95% of the terminologies and also improves translation quality on English-German benchmarks. Even when applied to lexically constrained MT output, our approach is able to improve preservation of the terminologies. However, we show that our models do not learn to copy constraints systematically and suggest a simple data augmentation technique that leads to improved performance and robustness.

Motivation & Objective

• Address the gap in automatic post-editing (APE) systems that fail to preserve domain-specific terminology constraints.
• Enable APE models to systematically retain user-specified lexical terms during post-editing, even when applied to already constrained MT outputs.
• Improve model robustness in handling rare or semantically divergent constraints such as synonyms and antonyms.
• Investigate the effectiveness of both autoregressive (Transformer) and non-autoregressive (Levenshtein Transformer) architectures for constrained APE.
• Develop and evaluate a data augmentation strategy to enhance systematic copying of constraints in APE models.

Proposed method

• Augment input sequences with constraint tokens and factors (e.g., source, constraint, target) to explicitly represent terminology constraints in the encoder input.
• Train autoregressive Transformer (MST) and non-autoregressive Levenshtein Transformer (LevT) models to generate post-edits that include specified terminology pairs.
• Use a constrained decoding strategy where the decoder is initialized with a partial output sequence containing the target-side terminology constraints.
• Apply data augmentation by replacing target-side constraints with synonyms or antonyms (using Wiktionary) to increase exposure to diverse constraint types during pretraining and fine-tuning.
• Evaluate model performance using standard APE metrics: BLEU, TER, and terminology preservation rate (Term%).
• Assess model stability by measuring TER and BLEU changes when replacing constraints with semantically divergent terms (synonyms/antonyms).

Experimental results

Research questions

• RQ1 Can constrained APE models preserve 95% of supplied terminology constraints while improving translation quality?
• RQ2 How does constrained APE compare to unconstrained APE in preserving terminology when applied to already terminology-constrained MT outputs?
• RQ3 Do APE models learn to systematically copy arbitrary terminology constraints, including synonyms and antonyms?
• RQ4 To what extent does data augmentation with synonym and antonym constraints improve model robustness and terminology preservation?
• RQ5 How stable are constrained APE models when faced with semantically divergent target-side constraints (e.g., synonyms vs. antonyms)?

Key findings

• Constrained APE models preserve over 95% of terminology constraints across both autoregressive and non-autoregressive models, with the LevT model achieving the highest coverage.
• Applying unconstrained APE on top of constrained MT leads to a 12.6% relative drop in terminology preservation, demonstrating the necessity of constraint-aware APE.
• Models fail to systematically copy unusual constraints such as synonyms and antonyms, with the MST model missing the antonym 'Verkleinerung' entirely.
• Data augmentation using synonyms and antonyms improves both translation quality (lower TER, higher BLEU) and terminology preservation, especially when applied during fine-tuning.
• Synonyms are harder to preserve than antonyms, likely due to stronger language model priors favoring original terms over semantically similar alternatives.
• Levenshtein Transformer models show higher TER scores under constraint variation, indicating lower stability compared to autoregressive models, suggesting trade-offs between speed and consistency.