[论文解读] Rethinking Explainability as a Dialogue: A Practitioner's Perspective

Himabindu Lakkaraju, Dylan Slack+3 more|arXiv (Cornell University)|Feb 3, 2022

Explainable Artificial Intelligence (XAI)被引用 28

一句话总结

本论文主张互动、自然语言的可解释性对话，并提供原则、设计路线图，以及旨在引导未来研究的从业者研究。

ABSTRACT

As practitioners increasingly deploy machine learning models in critical domains such as health care, finance, and policy, it becomes vital to ensure that domain experts function effectively alongside these models. Explainability is one way to bridge the gap between human decision-makers and machine learning models. However, most of the existing work on explainability focuses on one-off, static explanations like feature importances or rule lists. These sorts of explanations may not be sufficient for many use cases that require dynamic, continuous discovery from stakeholders. In the literature, few works ask decision-makers about the utility of existing explanations and other desiderata they would like to see in an explanation going forward. In this work, we address this gap and carry out a study where we interview doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Domain experts wish to treat machine learning models as "another colleague", i.e., one who can be held accountable by asking why they made a particular decision through expressive and accessible natural language interactions. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations as a starting place for future work. Further, we show why natural language dialogues satisfy these principles and are a desirable way to build interactive explanations. Next, we provide a design of a dialogue system for explainability and discuss the risks, trade-offs, and research opportunities of building these systems. Overall, we hope our work serves as a starting place for researchers and engineers to design interactive explainability systems.

研究动机与目标

• 了解领域专家（医疗保健和政策领域）在实践中如何使用模型解释。
• 识别当前可解释性方法的差距和痛点。
• 提出一套互动解释的原则。
• 概述可解释性对话系统的设计路线图。
• 讨论自然语言可解释性对话的风险、权衡与研究机会。

提出的方法

• 对医生和政策专家进行了26次半结构化访谈，以评估可解释性需求和痛点。
• 综合访谈发现，提取互动解释的原则。
• 提出一个用于可解释性对话系统的四模块架构（NLU、解释模块、响应生成、GUI）。
• 讨论脚本化与动态响应之间的设计选项与权衡，以及在多轮对话中处理上下文。
• 提供面向对话式解释的NLP、可解释性、UI和可扩展性方面的路线图与挑战。

实验结果

研究问题

• RQ1 从业者在静态特征重要性之外，对解释的需求和愿望是什么？
• RQ2 领域专家是否偏好互动、基于对话的解释，而非一次性解释？
• RQ3 在高风险领域，哪些原则应指引互动可解释性系统的设计？
• RQ4 自然语言可解释性对话系统的关键设计与实现挑战是什么？
• RQ5 在部署此类对话系统以实现可解释性方面的风险与机会是什么？

主要发现

• 领域专家对现有解释范式不满，渴望互动解释。
• 从业者更喜欢通过自然语言对话与模型交互，而非静态解释。
• 将自然语言对话视为更有效可解释性的有利路径。
• 从业者高度重视伴随解释的准确性/正确性指标。
• 提出五条互动解释的指导原则：适度互动、恰当回应、正确校准的响应、降低可解释性开销、考虑上下文。