[论文解读] Explanations Can Reduce Overreliance on AI Systems During Decision-Making

Helena Vasconcelos, Matthew Jörke+4 more|arXiv (Cornell University)|Dec 13, 2022

Decision-Making and Behavioral Economics被引用 20

一句话总结

本文表明AI解释可以在任务和解释成本在迷宫解题协作中与模拟AI一起被操控时减少过度依赖，采用五项研究（N=731）的成本-收益框架。

ABSTRACT

Prior work has identified a resilient phenomenon that threatens the performance of human-AI decision-making teams: overreliance, when people agree with an AI, even when it is incorrect. Surprisingly, overreliance does not reduce when the AI produces explanations for its predictions, compared to only providing predictions. Some have argued that overreliance results from cognitive biases or uncalibrated trust, attributing overreliance to an inevitability of human cognition. By contrast, our paper argues that people strategically choose whether or not to engage with an AI explanation, demonstrating empirically that there are scenarios where AI explanations reduce overreliance. To achieve this, we formalize this strategic choice in a cost-benefit framework, where the costs and benefits of engaging with the task are weighed against the costs and benefits of relying on the AI. We manipulate the costs and benefits in a maze task, where participants collaborate with a simulated AI to find the exit of a maze. Through 5 studies (N = 731), we find that costs such as task difficulty (Study 1), explanation difficulty (Study 2, 3), and benefits such as monetary compensation (Study 4) affect overreliance. Finally, Study 5 adapts the Cognitive Effort Discounting paradigm to quantify the utility of different explanations, providing further support for our framework. Our results suggest that some of the null effects found in literature could be due in part to the explanation not sufficiently reducing the costs of verifying the AI's prediction.

研究动机与目标

• 激励并测试解释是否能够通过将与AI的互动框定为成本-收益决策来缓解过度依赖。
• 将认知努力、任务难度和激励因素正式化为一个成本-收益框架，影响用户是否会参与AI解释。
• 研究不同任务难度、解释复杂性和货币激励如何影响对AI预测与解释的依赖。
• 证明在某些条件下解释可以减少过度依赖，挑战“解释永远无用”的看法。
• 使用认知努力折扣范式量化解释的效用，以连接成本、收益与解释有用性。

提出的方法

• 建立一个成本-收益框架，使用户在参与任务（有无验证AI）之间进行选择，或者直接依赖AI。
• 在五项研究中，使用带有模拟AI的迷宫解题任务来操纵任务难度、解释难度和货币激励。
• 采用两种解释模态（高亮显示和书面解释），并确保跨研究的AI准确度维持在80%。
• 针对AI预测正确时生成准确解释；当AI出错时生成误导性路径的解释，以研究它们对验证行为的影响。
• 衡量过度依赖、任务参与度和主观效用，并在研究5中辅以自我报告量表和认知努力折扣方法。

实验结果

研究问题

• RQ1 在何种条件下人们会参与AI解释以验证预测并减少过度依赖？
• RQ2 任务难度、解释难度和货币激励如何影响参与解释与否以對比依赖AI的效用？
• RQ3 成本-收益视角能否解释在某些情况下解释减少过度依赖而非无效的情形？

主要发现

• 随着任务难度的增加，解释对减少过度依赖的作用超过单独的预测。
• 理解解释所需的认知努力越低，过度依赖的降低越显著。
• 为准确性提供更高的货币奖励会降低过度依赖，表明收益会影响参与策略。
• 更易理解的解释获得更高的主观效用，增加参与任务的可能性。
• 改编的认知努力折扣范式证实，理解成本较低的解释更受重视，且任务难度提升解释的效用。