[Paper Review] Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling

Ilia Kulikov, Maksim Eremeev+1 more|arXiv (Cornell University)|Dec 16, 2021

Neural Networks and Applications4 citations

TL;DR

This paper introduces the oversmoothing rate to quantify how often neural autoregressive sequence models assign higher probability to overly short sequences (e.g., premature <eos> tokens) than to full reference sequences. By proposing a differentiable oversmoothing loss for training, the method effectively reduces this oversmoothing, leading to improved sequence length modeling and BLEU scores under large-beam decoding, especially in neural machine translation tasks.

ABSTRACT

Neural autoregressive sequence models smear the probability among many possible sequences including degenerate ones, such as empty or repetitive sequences. In this work, we tackle one specific case where the model assigns a high probability to unreasonably short sequences. We define the oversmoothing rate to quantify this issue. After confirming the high degree of oversmoothing in neural machine translation, we propose to explicitly minimize the oversmoothing rate during training. We conduct a set of experiments to study the effect of the proposed regularization on both model distribution and decoding performance. We use a neural machine translation task as the testbed and consider three different datasets of varying size. Our experiments reveal three major findings. First, we can control the oversmoothing rate of the model by tuning the strength of the regularization. Second, by enhancing the oversmoothing loss contribution, the probability and the rank of token decrease heavily at positions where it is not supposed to be. Third, the proposed regularization impacts the outcome of beam search especially when a large beam is used. The degradation of translation quality (measured in BLEU) with a large beam significantly lessens with lower oversmoothing rate, but the degradation compared to smaller beam sizes remains to exist. From these observations, we conclude that the high degree of oversmoothing is the main reason behind the degenerate case of overly probable short sequences in a neural autoregressive model.

Motivation & Objective

• To formally characterize the issue of oversmoothing in neural autoregressive sequence models, where models assign high probability to unreasonably short sequences.
• To define and quantify the oversmoothing rate as the frequency with which a ground-truth sequence is outscored by its own prefixes in probability.
• To propose a differentiable regularization loss (oversmoothing loss) that directly minimizes the oversmoothing rate during training.
• To evaluate the impact of this regularization on sequence length distribution, decoding behavior, and translation quality across multiple machine translation datasets.
• To investigate why beam search with large beam sizes degrades translation quality and whether oversmoothing is a root cause.

Proposed method

• Define the oversmoothing rate as the average probability that any prefix of the ground-truth sequence has higher probability than the full sequence.
• Construct a convex relaxation of the oversmoothing rate to create a differentiable loss function suitable for gradient-based optimization.
• Integrate the oversmoothing loss into the standard negative log-likelihood objective during training of neural machine translation models.
• Train models on IWSLT’17 and WMT translation tasks with varying regularization strengths (α) to control oversmoothing.
• Use beam search decoding with varying beam sizes to evaluate the effect on sequence length ratio and BLEU scores.
• Analyze the log-probability and rank of the <eos> token across decoding positions to study model behavior changes.

Figure 1: Average oversmoothing rate is going down as we increase contribution of the oversmoothing loss during fine-tuning. Filled regions denote the standard deviation across training runs according to section 5 .

Experimental results

Research questions

• RQ1 To what extent do well-trained neural machine translation models exhibit high oversmoothing rates, particularly in relation to premature <eos> token generation?
• RQ2 Can the oversmoothing rate be effectively controlled through a differentiable regularization loss during training?
• RQ3 How does reducing the oversmoothing rate affect the length distribution of generated sequences, especially under large-beam decoding?
• RQ4 Does the proposed regularization improve translation quality (measured by BLEU) when using large beam sizes, and if so, why?
• RQ5 Why does the improvement from reduced oversmoothing not manifest with small beam sizes, despite better length modeling?

Key findings

• The oversmoothing rate can be monotonically reduced by increasing the strength of the proposed regularization loss, demonstrating effective control over the issue.
• As the oversmoothing loss contribution increases, the probability and rank of the <eos> token at premature positions drop significantly, indicating reduced preference for short sequences.
• The proposed regularization dramatically reduces the degradation of length ratio with increasing beam size, maintaining a more consistent length distribution closer to reference sequences.
• Translation quality (BLEU) improves when decoding with large beam sizes under lower oversmoothing rates, though the improvement is not observed with small beam sizes.
• The dynamic range of the <eos> token's log-probability increases with regularization, and its rank is significantly altered, especially in longer sequences.
• Despite reducing oversmoothing, the performance gap between large and small beam sizes persists, suggesting beam search inefficiency remains unresolved.

Figure 2: (a) Log-probabilities of $\left<\mathrm{eos}\right>$ token within length- $t$ prefixes averaged across all positions per translation and then averaged across all translations. (b) Normalized rank of $\left<\mathrm{eos}\right>$ token within length- $t$ prefixes averaged across all positions