[论文解读] Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support

Tim Miller|arXiv (Cornell University)|Feb 24, 2023

Explainable Artificial Intelligence (XAI)被引用 10

一句话总结

该论文主张评估性 AI，一种以假设为驱动的决策支持框架，将从推荐驱动的可解释 AI 转向以人为中心、基于证据的方法，支持 abductive reasoning 并保持用户控制。

ABSTRACT

In this paper, we argue for a paradigm shift from the current model of explainable artificial intelligence (XAI), which may be counter-productive to better human decision making. In early decision support systems, we assumed that we could give people recommendations and that they would consider them, and then follow them when required. However, research found that people often ignore recommendations because they do not trust them; or perhaps even worse, people follow them blindly, even when the recommendations are wrong. Explainable artificial intelligence mitigates this by helping people to understand how and why models give certain recommendations. However, recent research shows that people do not always engage with explainability tools enough to help improve decision making. The assumption that people will engage with recommendations and explanations has proven to be unfounded. We argue this is because we have failed to account for two things. First, recommendations (and their explanations) take control from human decision makers, limiting their agency. Second, giving recommendations and explanations does not align with the cognitive processes employed by people making decisions. This position paper proposes a new conceptual framework called Evaluative AI for explainable decision support. This is a machine-in-the-loop paradigm in which decision support tools provide evidence for and against decisions made by people, rather than provide recommendations to accept or reject. We argue that this mitigates issues of over- and under-reliance on decision support tools, and better leverages human expertise in decision making.

研究动机与目标

• 促使从 recommendation-driven XAI 向 hypothesis-driven evaluative AI 的决策支持范式转变。
• 强调当前可解释性工具在实现稳健的人类决策方面的局限性。
• 提出一个与认知对齐的框架，保留人类主体性并改善信任校准。
• 概述 evaluative AI 的研究议程及其在高风险决策中的适用性。

提出的方法

• 批判性地回顾现有 XAI 范式（推荐驱动、对比性解释、本质可解释模型、认知强制）在决策制定标准上的表现。
• 将 evaluative AI 框架建立在 abductive reasoning 与 sensemaking 的 Data/Frame 理论之上。
• 定义 evaluative AI 的两个设计标准：支持认知决策过程并改善对选项探索的控制权。
• 提出一个两部分的概念框架，详细说明 evaluative AI 如何为/反映假设生成/组织证据并支持权衡取舍。
• 提供一个关于皮肤癌病变诊断的示意诊断界面示例，说明假设如何被筛选以及证据如何呈现。

实验结果

研究问题

• RQ1 当前基于推荐的 XAI 在现实世界决策中的局限性是什么？
• RQ2 决策支持工具如何与人类的 abductive reasoning 与 sensemaking 过程保持一致？
• RQ3 一个以假设为驱动的 evaluative AI 会是什么样子，如何支持选项、判断和权衡取舍？
• RQ4 在哪些情境下 evaluative AI 对决策最为有效？

主要发现

• 基于推荐的 XAI 常导致过度依赖或不足依赖以及对解释的参与度有限。
• abductive reasoning 与 sensemaking 是人们评估 AI 输出与假设的核心。
• evaluative AI 将 DST 的角色从给出推荐转变为提供对/反假设的证据并支持权衡取舍，从而提升用户控制。
• evaluative AI 框架更符合认知决策过程，能缓解固定化与自动化偏差。
• 一个示例诊断实例展示了 evaluative AI 如何呈现多种可行假设及其支持/反证证据。