[论文解读] Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task

Melissa Ailem, Jinghsu Liu+1 more|arXiv (Cornell University)|Nov 3, 2021

Natural Language Processing Techniques被引用 4

一句话总结

本论文介绍了 Lingua Custodia 在 WMT 2021 共享任务中针对受限神经机器翻译的解决方案，通过训练阶段的数据增强和约束标记掩码策略，提升了术语遵循度。该方法在英语-法语翻译中将精确匹配准确率显著提升至 91.9%，同时保持了较高的 BLEU（44.90）和 COMET（0.681）得分，优于标准 Transformer 模型和推理阶段约束解码方法。

ABSTRACT

This paper describes Lingua Custodia's submission to the WMT21 shared task on machine translation using terminologies. We consider three directions, namely English to French, Russian, and Chinese. We rely on a Transformer-based architecture as a building block, and we explore a method which introduces two main changes to the standard procedure to handle terminologies. The first one consists in augmenting the training data in such a way as to encourage the model to learn a copy behavior when it encounters terminology constraint terms. The second change is constraint token masking, whose purpose is to ease copy behavior learning and to improve model generalization. Empirical results show that our method satisfies most terminology constraints while maintaining high translation quality.

研究动机与目标

• 提升神经机器翻译模型在生物医学和公共卫生等专业领域中对术语约束的遵循能力。
• 通过利用单语数据进行数据增强，解决并行训练数据中术语覆盖率低的挑战。
• 通过一种新颖的掩码策略，增强模型在处理冲突术语映射时的泛化能力和鲁棒性。
• 开发一种训练阶段方法，避免推理阶段的计算开销，同时确保严格的术语合规性。

提出的方法

• 通过在源语言端的术语项上添加 <S> 和 </S> 标签来增强训练数据，以引导模型产生复制行为。
• 引入一种掩码标记策略，即在 <S> 标签后将源术语替换为 MASK 标记，以教会模型复制目标术语。
• 采用基于 Transformer 的架构，使用共享嵌入和 8 个注意力头，基于多语言数据进行训练，采用 BPE 或 SentencePiece 分词。
• 对英语-法语和英语-俄语使用 BPE 子词分词，合并次数为 40k；对英语-中文使用 SentencePiece，词汇表大小在 39,996 到 52,172 之间。
• 训练模型最多 100 个周期，基于验证损失进行早停，推理时使用 5 的束宽。
• 使用标准机器翻译指标（BLEU、COMET）和术语特定指标（精确匹配准确率、窗口重叠、1-TERm）对系统进行评估。

实验结果

研究问题

• RQ1 在神经机器翻译中，通过术语标记进行训练阶段数据增强是否能提升模型对词汇约束的遵循能力？
• RQ2 约束标记掩码策略是否能增强模型在处理冲突术语映射时的泛化能力和鲁棒性？
• RQ3 与推理阶段约束解码相比，所提出方法在术语准确率和翻译质量方面表现如何？
• RQ4 在多大程度上可以利用单语数据来增加包含术语项的训练样本数量？
• RQ5 该方法是否能在实现近乎完美的术语遵循度的同时，保持较高的翻译质量（BLEU、COMET）？

主要发现

• TAG+MASK 模型在英语-法语测试集上实现了 91.9% 的精确匹配准确率，显著优于标准 Transformer（32.5%）和约束解码器（85.6%）基线模型。
• 在英语-法语数据集上，该模型的 BLEU 得分为 44.90，COMET 得分为 0.681，表明其在保持高翻译质量的同时，术语合规性表现优异。
• 在英语-俄语任务中，系统实现了 84.9% 的精确匹配准确率和 29.13 的 BLEU 得分，显示出在词形丰富的语言上的强劲表现。
• 在英语-中文任务中，模型获得了 82.9% 的精确匹配准确率和 29.16 的 BLEU 得分，表明其在低资源、非拉丁字母脚本翻译中的有效适应能力。
• 窗口重叠（2）和（3）指标在所有语言对中均表现出一致提升，英语-法语任务中分别达到 34.4% 和 33.5% 的重叠率，表明术语项的上下文定位更加准确。
• 英语-法语任务中 1-TERm 得分为 0.598，表明与基线相比，术语相关编辑错误显著减少。