[论文解读] Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs

Myra Cheng, Robert D. Hawkins+1 more|arXiv (Cornell University)|Jan 7, 2026

Artificial Intelligence in Healthcare and Education被引用 0

一句话总结

论文认为大型语言模型在挑战有害信念方面失败，原因在于过度迁就与认知警觉性不足，并且展示了务实性提示可以显著提升安全基准表现。

ABSTRACT

Large language models (LLMs) frequently fail to challenge users' harmful beliefs in domains ranging from medical advice to social reasoning. We argue that these failures can be understood and addressed pragmatically as consequences of LLMs defaulting to accommodating users' assumptions and exhibiting insufficient epistemic vigilance. We show that social and linguistic factors known to influence accommodation in humans (at-issueness, linguistic encoding, and source reliability) similarly affect accommodation in LLMs, explaining performance differences across three safety benchmarks that test models' ability to challenge harmful beliefs, spanning misinformation (Cancer-Myth, SAGE-Eval) and sycophancy (ELEPHANT). We further show that simple pragmatic interventions, such as adding the phrase "wait a minute", significantly improve performance on these benchmarks while preserving low false-positive rates. Our results highlight the importance of considering pragmatics for evaluating LLM behavior and improving LLM safety.

研究动机与目标

• 通过务实视角的迁就与认知警觉性解释为何LLM在挑战有害信念方面失效。
• 识别影响LLM迁就的语言与社会因素（议题性、编码、信息源可靠性）。
• 证明简单的务实干预在不增加误报的情况下可以提升安全基准表现。
• 为基准设计与提示提供指导，以更好地评估和提升LLM的安全性。

提出的方法

• 在三个安全基准（Cancer-Myth、SAGE-Eval、ELEPHANT）中，复现人类务实因素对LLM迁就的影响。
• 操控议题性、语言编码（前提假设与断言）与信息源可靠性，研究其对LLM表现的影响。
• 在六种最先进的LLM上进行评估（三种闭源、三种开源）。
• 测试两种务实干预（显式纠正指令与话语标记“wait a minute”）在推断时改变认知警觉性。
• 提供回归分析和受控实验（如2x3和2x2设计）以量化因素对表现的影响。

Figure 1: Mean ( $\pm 95\%$ CI) benchmark scores by each factor (H1a-H1c). Higher is better for SAGE-Eval and Cancer-Myth, and lower is better for ELEPHANT (r/AITA and Subjective Statements). Each factor results in significantly higher overall performance in the expected direction for all factors. E

实验结果

研究问题

• RQ1 议题性、语言编码和信息源可靠性是否如同在人类中一样影响LLM的迁就？
• RQ2 简单的务实干预是否能够改变认知警觉性并在不提高误报的情况下提升LLM安全基准表现？
• RQ3 干预对不同基准（Cancer-Myth、SAGE-Eval、ELEPHANT）在多模型中有何影响？
• RQ4 在评估LLM安全性时，基准设计与提示设计有哪些含义？

主要发现

• 议题性是影响LLM在安全基准上表现的主导因素；前提下的非议题性内容更难纠正。
• 显式标注误解或加入“wait a minute”话语标记在有控制的误报率下显著提升各基准的表现。
• 信息源不可靠会削弱其他线索的影响，与人类认知警觉性的模式一致。
• 干预在六种LLM中的收益显著（如Cancer-Myth接近4倍提升；SAGE-Eval约40%提升）。
• 基准对务实提示的变化敏感，强调评估与设计中需考虑务实性因素。

Figure 2: Mean score ( $\pm$ 95% CI) of interventions on Cancer-Myth, SAGE-Eval and ELEPHANT. Higher is better for Cancer-Myth and SAGE-Eval, closer to 0 is better for ELEPHANT. We find that both interventions overall yield significant improvements for Cancer-Myth, validation and indirectness. For f