[论文解读] To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making

Zana Buçinca, Maja Barbara Malaya+1 more|arXiv (Cornell University)|Feb 19, 2021

Decision-Making and Behavioral Economics参考文献 66被引用 117

一句话总结

认知强制函数（CFF）在AI辅助决策中减少对AI的过度依赖，相较于简单的可解释AI方法，虽然它们对认知负担更高，但对Need for Cognition较高的人群收益更多。

ABSTRACT

People supported by AI-powered decision support tools frequently overrely on the AI: they accept an AI's suggestion even when that suggestion is wrong. Adding explanations to the AI decisions does not appear to reduce the overreliance and some studies suggest that it might even increase it. Informed by the dual-process theory of cognition, we posit that people rarely engage analytically with each individual AI recommendation and explanation, and instead develop general heuristics about whether and when to follow the AI suggestions. Building on prior research on medical decision-making, we designed three cognitive forcing interventions to compel people to engage more thoughtfully with the AI-generated explanations. We conducted an experiment (N=199), in which we compared our three cognitive forcing designs to two simple explainable AI approaches and to a no-AI baseline. The results demonstrate that cognitive forcing significantly reduced overreliance compared to the simple explainable AI approaches. However, there was a trade-off: people assigned the least favorable subjective ratings to the designs that reduced the overreliance the most. To audit our work for intervention-generated inequalities, we investigated whether our interventions benefited equally people with different levels of Need for Cognition (i.e., motivation to engage in effortful mental activities). Our results show that, on average, cognitive forcing interventions benefited participants higher in Need for Cognition more. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.

研究动机与目标

• 以双过程理论来解决决策中的AI过度依赖，提供动机。
• 研究认知强制干预是否能降低对AI解释的依赖。
• 将三种认知强制设计与简单的可解释AI方法以及无AI基线进行比较。
• 评估干预对认知需求（Need for Cognition，NFC）相关不平等的影响。
• 识别强制函数的有效性与用户可接受性之间的权衡。

提出的方法

• 设计并进行一项在线实验（N=199），将3种认知强制设计与2个SXAI条件及无AI基线进行比较。
• 使用基于营养的任务，让参与者在AI解释的引导下用低碳替代品替换高碳水化合物成分。
• 实现一个识别准确度为75%的模拟AI，并提供四个顶级替代选项，附带基于特征的解释（碳水减少和风味相似性）。
• 引入六种条件：No AI、SXAI（Explanation, Uncertainty）以及CFF（On demand, Update, Wait）。
• 测量客观结果（总体表现、碳源检测、碳水减少、风味相似性、过度依赖、人为错误）和主观指标（偏好、信任、认知负荷、系统复杂性）。
• 使用混合效应模型、Holm-Bonferroni事后检验以及用于NFC调节的Pearson相关分析进行分析。

(a) explanation (SXAI)

实验结果

研究问题

• RQ1 当AI预测错误时，认知强制是否能相对于SXAI方法降低对AI的过度依赖？
• RQ2 认知强制干预是否提升总体任务表现及与最佳营养替代的对齐程度？
• RQ3 干预有效性与用户可接受性之间存在哪些权衡？
• RQ4 NFC水平是否调节认知强制函数的收益，可能导致结果的不平等？

主要发现

• CFF显著降低对AI的过度依赖，并在AI预测错误时提高客观性能，相较于SXAI。
• 在所有情境中，SXAI和CFF均优于无AI基线，提升了性能。
• 当AI预测错误时，CFF带来更高的碳源检测、碳水减少和风味相似性，以及更正确的决策数量，优于SXAI。
• 存在权衡：用户对CFF的可接受性较低，因为其需要更高的认知努力。
• 干预产生的不平等审计显示，NFC水平更高的个体平均从CFF中获益更多。
• 总体而言，认知动机等人因因素影响可解释AI策略的有效性。

(b) uncertainty (SXAI)