[论文解读] Apply Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level

Shaohui Kuang, Lifeng Han|arXiv (Cornell University)|May 3, 2018

Natural Language Processing Techniques被引用 2

一句话总结

本文提出将汉字部首整合入神经机器翻译（NMT）中，以改善中文到英文翻译中的未登录词（OOV）处理及翻译充分性。通过将部首作为语义单元与字符并列建模，该方法在BLEU、NIST、LEPOR、BEER和CharacTER等多个指标上均提升了性能，尤其在保留词边界知识时，充分性方面取得了显著提升。

ABSTRACT

In neural machine translation (NMT), researchers face the challenge of un-seen (or out-of-vocabulary OOV) words translation. To solve this, some researchers propose the splitting of western languages such as English and German into sub-words or compounds. In this paper, we try to address this OOV issue and improve the NMT adequacy with a harder language Chinese whose characters are even more sophisticated in composition. We integrate the Chinese radicals into the NMT model with different settings to address the unseen words challenge in Chinese to English translation. On the other hand, this also can be considered as semantic part of the MT system since the Chinese radicals usually carry the essential meaning of the words they are constructed in. Meaningful radicals and new characters can be integrated into the NMT systems with our models. We use an attention-based NMT system as a strong baseline system. The experiments on standard Chinese-to-English NIST translation shared task data 2006 and 2008 show that our designed models outperform the baseline model in a wide range of state-of-the-art evaluation metrics including LEPOR, BEER, and CharacTER, in addition to the traditional BLEU and NIST scores, especially on the adequacy-level translation. We also have some interesting findings from the results of our various experiment settings about the performance of words and characters in Chinese NMT, which is different with other languages. For instance, the full character level NMT may perform very well or the state of the art in some other languages as researchers demonstrated recently, however, in the Chinese NMT model, word boundary knowledge is important for the model learning.

研究动机与目标

• 解决中文到英文神经机器翻译中的未登录词（OOV）挑战。
• 探究汉字部首——作为字符的语义组成部分——是否能超越字符级建模，提升翻译质量。
• 评估词边界知识在中文NMT中的作用，与其它语言中的发现进行对比。
• 通过利用嵌入在部首中的语义信息，提升翻译充分性。
• 证明部首感知建模能在多个评估指标上带来一致的性能提升。

提出的方法

• 将汉字部首作为额外输入表征整合进NMT编码器-解码器框架。
• 采用基于注意力机制的NMT模型作为强基线进行对比。
• 设计多种模型变体，在不同层级整合部首信息：字符级、子字符级以及结合部首特征的词级。
• 在NIST中文到英文翻译基准数据集（2006年和2008年）上训练模型。
• 采用联合嵌入空间，对部首和字符进行编码，以捕捉语义与结构关系。
• 应用注意力机制，使解码器在翻译过程中可关注相关部首与字符。

实验结果

研究问题

• RQ1 在NMT中整合汉字部首是否能提升对未见或未登录词的翻译性能？
• RQ2 与标准字符级建模相比，部首整合如何影响翻译充分性？
• RQ3 与其它语言相比，词边界知识在中文NMT中是否具有更重要的作用？
• RQ4 部首是否作为有效的语义单元，提升模型对表面字符形式之外的泛化能力？
• RQ5 在BLEU、NIST、LEPOR、BEER和CharacTER得分方面，不同整合策略（如字符级与部首增强）的对比表现如何？

主要发现

• 部首增强的NMT模型在所有主要评估指标上均优于基线模型，包括BLEU、NIST、LEPOR、BEER和CharacTER。
• 模型在翻译充分性方面表现尤为突出，表明语义保真度更高。
• 完整字符级NMT模型表现良好，但引入部首后带来了持续性提升，尤其在罕见或未登录词上。
• 结果表明，词边界知识在中文NMT中对有效学习至关重要，这与其它语言中字符级模型可能已足够的发现形成对比。
• 研究揭示，部首携带有意义的语义信息，增强了模型对未见字符与词语的泛化能力。
• 不同实验设置表明，基于部首的建模在捕捉中文的形态与语义结构方面，优于纯字符级建模。