[论文解读] Improving Translation Faithfulness of Large Language Models via Augmenting Instructions

Yijie Chen, Yijin Liu+4 more|arXiv (Cornell University)|Aug 24, 2023

Topic Modeling被引用 6

一句话总结

本文提出SWIE（分段加权指令嵌入）和OverMiss——一种对比性指令微调数据集，以提升大语言模型在翻译任务中的忠实度。SWIE通过分段加权嵌入增强模型对指令的关注，而OverMiss则针对过度翻译和漏译错误进行优化。该方法在BLOOMZ和LLaMA模型的零样本、长上下文及少样本设置下，显著提升了BLEU分数（最高提升+0.56）与忠实度指标。

ABSTRACT

Large Language Models (LLMs) present strong general capabilities, and a current compelling challenge is stimulating their specialized capabilities, such as machine translation, through low-cost instruction tuning. The standard instruction-following data is sequentially organized as the concatenation of an instruction, an input, and a response. As the attention mechanism of LLMs has limitations on local focus, LLMs tend to focus more on the words or sentences nearby at each position. This leads to a high risk of instruction forgetting during decoding. To alleviate the above issues, We propose SWIE (Segment-Weighted Instruction Embedding) and an instruction-following dataset OVERMISS. SWIE improves the model instruction understanding by adding a global instruction representation on the following input and response representations. OVERMISS improves model faithfulness by comparing over-translation and miss-translation results with the correct translation. We apply our methods to two main-stream open-source LLMs, BLOOM and LLaMA. The experimental results demonstrate significant improvements in translation performance with SWIE based on BLOOMZ-3b, particularly in zero-shot and long text translations due to reduced instruction forgetting risk. Additionally, OVERMISS outperforms the baseline in translation performance (e.g. an increase in BLEU scores from 0.69 to 3.12 and an average improvement of 0.48 percentage comet scores for LLaMA-7b) with further enhancements seen in models combining OVERMISS and SWIE (e.g. the BLUE scores increase up to 0.56 from English to German across three different backbones), and both exhibit improvements in the faithfulness metric based on word alignment.

研究动机与目标

• 为解决因果语言模型在翻译解码过程中因局部注意力偏差导致的指令遗忘问题。
• 通过减少幻觉现象，特别是过度翻译与漏译，提升翻译忠实度。
• 开发一种数据构建方法，生成用于提升忠实度的对比性负样本。
• 在多种主干模型与翻译基准上验证SWIE与OverMiss的有效性。
• 分析注意力分布并证明SWIE可增强中层注意力对指令的关注，尤其在中层。

提出的方法

• SWIE引入参数化适配器，将全局指令表征注入输入与响应表征中，并通过分段加权注意力突出指令标记。
• 该方法采用跨层注意力分数的加权和，重点增强对指令段相对于输入与输出段的关注。
• OverMiss是一种新颖的指令微调数据集，通过显式提示模型避免过度翻译与漏译，构建对比性负样本。
• 数据集包含如“禁止过度翻译”和“禁止漏译”等指令，以引导模型生成忠实输出。
• 忠实度通过跨语言词对齐计算源语言与目标语言的词覆盖率来量化，使用平均召回率作为指标。
• 可视化模型各层的内部注意力分数，对比基线模型与SWIE模型，显示SWIE在指令结尾处注意力显著增强。

实验结果

研究问题

• RQ1 通过分段加权嵌入增强对指令的关注，是否能减少因果大语言模型在翻译过程中对指令的遗忘？
• RQ2 使用OverMiss进行对比性指令微调，能否有效减少过度翻译与漏译错误？
• RQ3 SWIE如何影响模型各层的注意力分布，特别是在最可能发生指令遗忘的中层？
• RQ4 SWIE与OverMiss在零样本与长上下文翻译任务中，对翻译性能与忠实度的提升程度如何？
• RQ5 SWIE与OverMiss的结合是否在BLEU与忠实度指标上产生协同增益？

主要发现

• SWIE使BLOOMZ-3b在四个WMT22翻译方向上的BLEU分数从0.19提升至0.51，六组零样本方向的BLEU分数从0.20提升至0.58。
• 在长句翻译任务中，BLOOMZ-3b使用SWIE后平均BLEU得分为0.67，表明在长上下文设置下性能显著提升。
• OverMiss使WMT22测试集的BLEU分数从0.69提升至3.12，LLaMA-7b的平均COMET得分提升0.48分。
• SWIE与OverMiss结合在三种主干模型的英德翻译任务中，BLEU最高提升0.56。
• 通过词对齐进行的忠实度评估显示，OverMiss使得分从87.94（Parrot-hint）提升至88.84，两者结合后达88.80。
• 注意力可视化结果证实，SWIE使中层指令结尾处的注意力比例最高提升20%，验证了指令遗忘的减少。