[Paper Review] Prevent the Language Model from being Overconfident in Neural Machine Translation

Mengqi Miao, Fandong Meng+3 more|arXiv (Cornell University)|May 24, 2021

Natural Language Processing Techniques43 references4 citations

TL;DR

This paper proposes a novel approach to reduce hallucination in Neural Machine Translation (NMT) by addressing the overconfidence of the intrinsic language model (LM) within NMT. It introduces the Margin between NMT and LM as a measure of LM overconfidence and proposes two objectives—MTO and MSO—to maximize this margin, leading to significant BLEU gains of +1.36, +1.50, and +0.63 on WMT14 En-De, WMT19 Zh-En, and WMT14 En-Fr tasks, respectively, with improved adequacy confirmed by human evaluation.

ABSTRACT

The Neural Machine Translation (NMT) model is essentially a joint language model conditioned on both the source sentence and partial translation. Therefore, the NMT model naturally involves the mechanism of the Language Model (LM) that predicts the next token only based on partial translation. Despite its success, NMT still suffers from the hallucination problem, generating fluent but inadequate translations. The main reason is that NMT pays excessive attention to the partial translation while neglecting the source sentence to some extent, namely overconfidence of the LM. Accordingly, we define the Margin between the NMT and the LM, calculated by subtracting the predicted probability of the LM from that of the NMT model for each token. The Margin is negatively correlated to the overconfidence degree of the LM. Based on the property, we propose a Margin-based Token-level Objective (MTO) and a Margin-based Sentencelevel Objective (MSO) to maximize the Margin for preventing the LM from being overconfident. Experiments on WMT14 English-to-German, WMT19 Chinese-to-English, and WMT14 English-to-French translation tasks demonstrate the effectiveness of our approach, with 1.36, 1.50, and 0.63 BLEU improvements, respectively, compared to the Transformer baseline. The human evaluation further verifies that our approaches improve translation adequacy as well as fluency.

Motivation & Objective

• To investigate the root cause of inadequate translations in NMT, particularly the overconfidence of the intrinsic language model (LM) in generating fluent but incorrect outputs.
• To define a measurable indicator of LM overconfidence during NMT decoding, enabling targeted intervention.
• To propose optimization objectives that directly maximize the margin between NMT and LM probabilities to reduce overreliance on partial translations.
• To improve translation adequacy without requiring additional data or model architecture changes, focusing on training-time optimization.

Proposed method

• The Margin between NMT and LM is defined as the difference between the NMT model’s predicted probability and the LM’s predicted probability for each target token.
• The Margin-based Token-level Objective (MTO) is formulated to maximize this margin at the token level during training.
• A dynamic weight function is introduced in the Margin-based Sentence-level Objective (MSO) to down-weight sentences with many negative-margin tokens, which are identified as 'dirty data' that harm model performance.
• The MSO objective is applied during fine-tuning to iteratively filter out problematic training examples based on the model’s own assessment of margin quality.
• The method is applied to the Transformer architecture without architectural modifications, using only loss function adjustments.
• The approach is trained end-to-end with standard cross-entropy loss, with the margin objectives added as auxiliary training signals.

Experimental results

Research questions

• RQ1 To what extent does the overconfidence of the intrinsic language model contribute to inadequate translations in NMT?
• RQ2 Can the margin between NMT and LM probabilities serve as a reliable indicator of LM overconfidence during decoding?
• RQ3 Does maximizing this margin through auxiliary objectives improve translation adequacy without sacrificing fluency?
• RQ4 Can a dynamic data selection mechanism based on margin quality enhance training efficiency and model generalization?

Key findings

• The proposed Margin-based Token-level Objective (MTO) achieved a +1.36 BLEU improvement on the WMT14 English-to-German translation task compared to the Transformer baseline.
• The Margin-based Sentence-level Objective (MSO) achieved a +1.50 BLEU improvement on the WMT19 Chinese-to-English translation task, outperforming MTO.
• On the WMT14 English-to-French task, the method achieved a +0.63 BLEU improvement, demonstrating consistent gains across diverse language pairs.
• Human evaluation confirmed that translations generated using the proposed method are significantly more adequate while maintaining or improving fluency.
• The dynamic weight function in MSO successfully identified and down-weighted 'dirty data' with high negative-margin content, improving training stability and performance.
• The method improves translation quality without architectural changes or external data, relying solely on loss function optimization.