[论文解读] Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling

Ilia Kulikov, Maksim Eremeev+1 more|arXiv (Cornell University)|Dec 16, 2021

Neural Networks and Applications被引用 4

一句话总结

本文引入了过平滑率（oversmoothing rate）来量化神经自回归序列模型将更高概率分配给过度简短序列（例如过早生成的<eos>标记）而非完整参考序列的频率。通过提出一种可微分的过平滑损失用于训练，该方法有效减少了这种过平滑现象，从而在大束宽解码下显著改善了序列长度建模与BLEU得分，尤其在神经机器翻译任务中表现突出。

ABSTRACT

Neural autoregressive sequence models smear the probability among many possible sequences including degenerate ones, such as empty or repetitive sequences. In this work, we tackle one specific case where the model assigns a high probability to unreasonably short sequences. We define the oversmoothing rate to quantify this issue. After confirming the high degree of oversmoothing in neural machine translation, we propose to explicitly minimize the oversmoothing rate during training. We conduct a set of experiments to study the effect of the proposed regularization on both model distribution and decoding performance. We use a neural machine translation task as the testbed and consider three different datasets of varying size. Our experiments reveal three major findings. First, we can control the oversmoothing rate of the model by tuning the strength of the regularization. Second, by enhancing the oversmoothing loss contribution, the probability and the rank of token decrease heavily at positions where it is not supposed to be. Third, the proposed regularization impacts the outcome of beam search especially when a large beam is used. The degradation of translation quality (measured in BLEU) with a large beam significantly lessens with lower oversmoothing rate, but the degradation compared to smaller beam sizes remains to exist. From these observations, we conclude that the high degree of oversmoothing is the main reason behind the degenerate case of overly probable short sequences in a neural autoregressive model.

研究动机与目标

• 正式刻画神经自回归序列模型中过平滑问题的本质，即模型对不合理短序列赋予过高概率。
• 定义并量化过平滑率，即真实序列在概率上被其自身前缀超越的频率。
• 提出一种可微分的正则化损失（过平滑损失），在训练过程中直接最小化过平滑率。
• 评估该正则化对序列长度分布、解码行为及多个机器翻译数据集上翻译质量的影响。
• 探究大束宽束搜索为何会降低翻译质量，以及过平滑是否为根本原因。

提出的方法

• 将过平滑率定义为任意真实序列前缀的概率高于完整序列的平均概率。
• 构建过平滑率的凸松弛形式，以生成适用于梯度优化的可微分损失函数。
• 在神经机器翻译模型训练过程中，将过平滑损失整合到标准负对数似然目标中。
• 在IWSLT’17和WMT翻译任务上训练模型，采用不同正则化强度（α）以控制过平滑程度。
• 使用不同束宽的束搜索解码，评估其对序列长度比和BLEU得分的影响。
• 分析解码过程中<eos>标记的对数概率及其排名，以研究模型行为的变化。

Figure 1: Average oversmoothing rate is going down as we increase contribution of the oversmoothing loss during fine-tuning. Filled regions denote the standard deviation across training runs according to section 5 .

实验结果

研究问题

• RQ1 经过充分训练的神经机器翻译模型在多大程度上表现出高过平滑率，尤其是在过早生成<eos>标记方面？
• RQ2 能否通过训练过程中引入可微分正则化损失有效控制过平滑率？
• RQ3 降低过平滑率如何影响生成序列的长度分布，特别是在大束宽解码下？
• RQ4 所提出的正则化是否能提升大束宽解码下的翻译质量（以BLEU衡量）？如果是，原因是什么？
• RQ5 为何降低过平滑后，性能提升在小束宽下未显现，尽管长度建模已更优？

主要发现

• 随着所提正则化损失强度的增加，过平滑率可单调降低，表明该问题得到了有效控制。
• 随着过平滑损失贡献的增强，过早位置处<eos>标记的概率与排名显著下降，表明模型对短序列的偏好减弱。
• 所提出的正则化显著减少了束宽增大时长度比的退化，使长度分布更稳定，更接近参考序列。
• 在低过平滑率下，使用大束宽解码时翻译质量（BLEU）得到提升，但小束宽下未见改善。
• 正则化使<eos>标记对数概率的动态范围增大，其排名显著改变，尤其在长序列中更为明显。
• 尽管降低了过平滑，大束宽与小束宽之间的性能差距依然存在，表明束搜索效率低下问题仍未解决。

Figure 2: (a) Log-probabilities of $\left<\mathrm{eos}\right>$ token within length- $t$ prefixes averaged across all positions per translation and then averaged across all translations. (b) Normalized rank of $\left<\mathrm{eos}\right>$ token within length- $t$ prefixes averaged across all positions