[论文解读] How Susceptible are LLMs to Influence in Prompts?

Sotiris Anagnostidis, Jannis Bulian|arXiv (Cornell University)|Aug 17, 2024

Artificial Intelligence in Law被引用 4

一句话总结

本研究调查了大型语言模型（LLMs）在受到其他模型的预测和解释等增强输入影响时的易感性。通过在多种问答任务中使用 Llama2、Mixtral 和 Falcon 模型，研究发现 LLMs 极易受到外部输入的影响，即使解释内容在事实上是错误的，尤其当输入被呈现为权威或高度自信时，这凸显了在检索增强和多智能体系统中存在风险。

ABSTRACT

Large Language Models (LLMs) are highly sensitive to prompts, including additional context provided therein. As LLMs grow in capability, understanding their prompt-sensitivity becomes increasingly crucial for ensuring reliable and robust performance, particularly since evaluating these models becomes more challenging. In this work, we investigate how current models (Llama, Mixtral, Falcon) respond when presented with additional input from another model, mimicking a scenario where a more capable model -- or a system with access to more external information -- provides supplementary information to the target model. Across a diverse spectrum of question-answering tasks, we study how an LLM's response to multiple-choice questions changes when the prompt includes a prediction and explanation from another model. Specifically, we explore the influence of the presence of an explanation, the stated authoritativeness of the source, and the stated confidence of the supplementary input. Our findings reveal that models are strongly influenced, and when explanations are provided they are swayed irrespective of the quality of the explanation. The models are more likely to be swayed if the input is presented as being authoritative or confident, but the effect is small in size. This study underscores the significant prompt-sensitivity of LLMs and highlights the potential risks of incorporating outputs from external sources without thorough scrutiny and further validation. As LLMs continue to advance, understanding and mitigating such sensitivities will be crucial for their reliable and trustworthy deployment.

研究动机与目标

• 了解 LLMs 在问答任务中对其他模型提供的增强输入的响应方式。
• 调查外部输入的质量、权威性或置信度水平是否会影响 LLM 的响应。
• 评估提示策略是否能减轻来自外部源的不当影响。
• 评估最先进开源 LLMs 在多样化推理和知识任务中对提示的敏感程度。
• 强调在未经过验证的情况下整合外部模型输出时，错误传播和偏见强化的风险。

提出的方法

• 使用三种开源 LLMs（Llama2、Mixtral 和 Falcon）进行实验。
• 在八个多样化的问答基准上评估模型响应：PIQA、SIQA、CommonsenseQA、OpenBookQA、WikiQA、GPQA、QuALITY 和 BoolQ。
• 在提示中引入一个外部模型（倡导者），向目标模型（裁判者）提供预测答案和推理过程。
• 改变三个关键影响因素：是否包含解释、来源权威性（五个等级）以及声明的置信度水平。
• 测量当裁判模型暴露于倡导者输入与基线提示时，其响应的变化。
• 使用受控提示评估缓解策略是否能降低易感性。

Figure 1: Leveraging external evidence, augmented inputs can help LLMs provide more informed answers.

实验结果

研究问题

• RQ1 外部模型解释的引入在多大程度上影响目标 LLM 对选择题的响应？
• RQ2 外部来源的感知权威性在多大程度上影响目标 LLM 的决策过程？
• RQ3 外部输入的声明置信度是否加剧或减轻目标模型的易感性？
• RQ4 提示策略能否有效减少外部输入对 LLM 输出的影响？
• RQ5 解释的正确性在多大程度上影响目标模型的响应，尤其是在解释内容在事实上错误时？

主要发现

• LLMs 极易受到外部模型输入的影响，即使所提供的解释在事实上是错误的。
• 无论解释的质量或逻辑有效性如何，解释的存在都会增加易感性。
• 当输入被标记为权威或高度自信时，模型更可能采纳所倡导的答案，尽管影响程度较小。
• 感知权威性和声明置信度水平越高，影响程度越大，表明模型对来源可信度线索敏感。
• 即使裁判模型的基线响应非常自信，也未能完全抵御外部影响。
• 标准提示技术不足以缓解这种易感性，表明需要更深层次的架构或推理层面解决方案。

Figure 3: Influence of advocates’ responses to judges’ predictions. Shading indicates whether an advocate provides their argument why their choice is the correct one, as seen in Tab. 2 (bottom right).