[论文解读] The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
自回归大语言模型在 A 是 B 上训练后未能推广到 B 是 A;逆向泛化近乎为零,且数据增强也无改善,在不同模型规模和家族中均如此。
We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Valentina Tereshkova was the first woman to travel to space", it will not automatically be able to answer the question, "Who was the first woman to travel to space?". Moreover, the likelihood of the correct answer ("Valentina Tershkova") will not be higher than for a random name. Thus, models do not generalize a prevalent pattern in their training set: if "A is B" occurs, "B is A" is more likely to occur. It is worth noting, however, that if "A is B" appears in-context, models can deduce the reverse relationship. We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. Code available at: https://github.com/lukasberglund/reversal_curse.
研究动机与目标
- 证明自回归大语言模型不能从“A 是 B”推广到“B 是 A”(逆向诅咒)。
- 表明这一逆向失败在不同模型规模与家族中仍然存在,且不能被数据增强所改善。
- 在合成微调数据和现实世界名人事实上评估该现象,以评估实际影响。
提出的方法
- 在形如“<name> 是 <description>”的合成事实上微调GPT-3和Llama-1模型,并测试对“<description> 是 <name>”的逆向泛化。
- 评估两种提示格式:在描述给出时的精确匹配准确度,以及正确名称在提供描述时的更高概率。
- 通过同义改写和“Both orders”子集来增强数据以促进元学习,然后比较结果。
实验结果
研究问题
- RQ1自回归大语言模型在对合成数据进行微调后,是否能从“A 是 B”推广到反向“B 是 A”?
- RQ2逆向诅咒是否可在现实世界知识中观测到,如名人父母-子女关系?
- RQ3数据增强或混合顺序训练能否缓解逆向失败?
- RQ4逆向效应是否因模型规模或家族(GPT-3、Llama 等)而异?
主要发现
- 当测试提示与微调顺序匹配时模型会泛化,但在反向方向完全失败(准确率接近零)。
- 对于 DescriptionToName 事实,GPT-3-175B 的正向精确匹配为 96.7%,反向为 0.1%。
- 对于 NameToDescription 事实,正向 50.0%,反向 0.0%。
- 提高可能性测试显示在顺序颠倒时,正确名称的对数概率与随机名称之间没有可检测的差异。
- 实验2 使用现实世界名人显示显著的不对称性:GPT-4 对父母相关问题回答良好,但难以从父母识别出名人,提示逆向诅咒的实际表现。
阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。