[论文解读] Quantifying Interpretability and Trust in Machine Learning Systems

Philipp Schmidt, Felix Bießmann|arXiv (Cornell University)|Jan 20, 2019

Explainable Artificial Intelligence (XAI)参考文献 6被引用 71

一句话总结

本文提出定量指标来衡量ML决策的可解释性和可信度，并通过众包实验和任务展示它们的应用，表明可解释性能提升人类生产力，同时揭示偏见信任。

ABSTRACT

Decisions by Machine Learning (ML) models have become ubiquitous. Trusting these decisions requires understanding how algorithms take them. Hence interpretability methods for ML are an active focus of research. A central problem in this context is that both the quality of interpretability methods as well as trust in ML predictions are difficult to measure. Yet evaluations, comparisons and improvements of trust and interpretability require quantifiable measures. Here we propose a quantitative measure for the quality of interpretability methods. Based on that we derive a quantitative measure of trust in ML decisions. Building on previous work we propose to measure intuitive understanding of algorithmic decisions using the information transfer rate at which humans replicate ML model predictions. We provide empirical evidence from crowdsourcing experiments that the proposed metric robustly differentiates interpretability methods. The proposed metric also demonstrates the value of interpretability for ML assisted human decision making: in our experiments providing explanations more than doubled productivity in annotation tasks. However unbiased human judgement is critical for doctors, judges, policy makers and others. Here we derive a trust metric that identifies when human decisions are overly biased towards ML predictions. Our results complement existing qualitative work on trust and interpretability by quantifiable measures that can serve as objectives for further improving methods in this field of research.

研究动机与目标

• 促使在ML决策中实现可衡量的可解释性和可信度的需求。
• 提出一个用于可解释性方法质量的定量指标。
• 推导一个信任指标，以识别受ML预测影响而产生偏见的人类决策。

提出的方法

• 基于信息传递速率的度量，捕捉在给出解释时人类多大程度上能复现ML模型的预测。
• 使用众包实验评估不同可解释性方法对人类理解和表现的影响。
• 证明提供解释可以在标注任务中使生产力提高超过一倍。
• 推导一个信任指标，以检测人类判断在多大程度上偏向ML预测。

实验结果

研究问题

• RQ1 定量测量能否可靠地区分可解释性强的解释与较不易解释的解释？
• RQ2 可解释性是否提升在人类决策中在ML辅助任务中的效率和准确性？
• RQ3 在人类显现出对ML预测的偏向信任的条件下，如何量化？

主要发现

• 提出的信息传递速率度量在众包研究中稳健地区分了可解释性方法。
• 解释显著提高标注任务的生产力（超过翻倍）。
• 推导出的信任指标能够识别在人类决策中对ML预测的偏向信任时刻。