[论文解读] The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models

Shuo Zhang, Liangming Pan+2 more|arXiv (Cornell University)|May 23, 2023

Topic Modeling被引用 5

一句话总结

本文提出了知识对齐（Knowledge Alignment）问题，即大型语言模型（LLMs）因约束和条件不匹配，无法正确将用户问题与外部知识关联。MixAlign通过结合基于LLM的约束匹配与针对性的人工澄清，将幻觉减少高达27.1%，并通过高质量、以用户为中心的澄清将性能提升22.2%。

ABSTRACT

Large language models often necessitate grounding on external knowledge to generate faithful and reliable answers. Yet even with the correct groundings in the reference, they can ignore them and rely on wrong groundings or their inherent biases to hallucinate when users, being largely unaware of the specifics of the stored information, pose questions that might not directly correlate with the retrieved groundings. In this work, we formulate this knowledge alignment problem and introduce MixAlign, a framework that interacts with both the human user and the knowledge base to obtain and integrate clarifications on how the user question relates to the stored information. MixAlign employs a language model to achieve automatic knowledge alignment and, if necessary, further enhances this alignment through human user clarifications. Experimental results highlight the crucial role of knowledge alignment in boosting model performance and mitigating hallucination, with improvements noted up to 22.2% and 27.1% respectively. We also demonstrate the effectiveness of MixAlign in improving knowledge alignment by producing high-quality, user-centered clarifications.

研究动机与目标

• 识别并形式化知识对齐问题，即LLM因隐含或缺失的约束，导致用户问题与外部知识错位。
• 弥补现有LLM系统依赖有偏见的内部知识而非忠实对齐外部知识的缺陷。
• 设计一种人机协同框架，通过自动化LLM推理与针对性人工澄清相结合，提升对齐效果。
• 评估知识对齐在减少幻觉和提升问答任务事实一致性方面的有效性。

提出的方法

• MixAlign以基于LLM的知识对齐启动，将用户问题中的约束映射到外部知识库中的对应字段。
• 当约束匹配存在不确定性时，MixAlign会生成针对性的澄清问题提交给人工用户。
• 该框架整合模型生成与人工提供的澄清信息，以在生成答案前优化知识定位。
• 采用两阶段流程：先进行显式约束匹配，再对剩余模糊情况实施隐式对齐。
• 系统根据约束匹配的置信度阈值动态决定是否需要用户介入。
• 设计上力求最小化用户负担，每实例平均仅需1.08–1.19轮澄清。

Figure 1: Knowledge Misalignment. Even if the user knows about the city constraint, he/she may not put it in the question being unaware that “city” is needed to filter noisy candidates. For the same reason, the user may not give a precise event name. Due to the misalignment between the user question

实验结果

研究问题

• RQ1 用户问题与外部知识之间的错位在多大程度上导致LLM产生幻觉？
• RQ2 仅靠基于LLM的约束匹配在多大程度上能解决知识对齐问题？
• RQ3 人工协助澄清对减少幻觉和提升事实准确性有何影响？
• RQ4 MixAlign框架在保持高对齐质量的同时，如何高效地最小化用户投入？
• RQ5 在不同噪声或无关知识基底条件下，MixAlign与理想化澄清及基线方法的性能表现如何比较？

主要发现

• 与基线模型相比，MixAlign将幻觉减少高达27.1%，凸显了知识对齐在提升事实一致性中的关键作用。
• 采用MixAlign后性能最高提升22.2%，表明将用户意图与外部知识对齐具有显著优势。
• 随着无关知识增多，MixAlign与仅LLM基线之间的性能差距扩大，覆盖度差距达7.7%，幻觉差距达9.8%，凸显了澄清的必要性。
• 人工协助至关重要：无人工版本在噪声增加时性能显著下降，表明LLM在自信约束匹配方面存在困难。
• 与朴素方法相比，MixAlign将用户澄清需求减少14.2%至22.5%，且每实例通常仅需1.08–1.19轮澄清。
• 隐式知识对齐不可或缺：若移除该机制，性能持续下降，且在噪声增加时差距保持稳定，表明显式对齐的错误会持续传播。

Figure 2: Oracle clarification results regarding different knowledge alignment types.