[论文解读] Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions

Milo Honegger|arXiv (Cornell University)|Aug 15, 2018

Explainable Artificial Intelligence (XAI)参考文献 22被引用 36

一句话总结

本论文提出了一套公理化框架，用于评估黑箱机器学习模型中个体预测的解释方法质量。通过将局部可解释性方法的理想特性形式化为一组公理，该框架实现了对解释质量的系统性评估与比较，为实践中评估模型可解释性提供了原则性基础。

ABSTRACT

From self-driving vehicles and back-flipping robots to virtual assistants who book our next appointment at the hair salon or at that restaurant for dinner - machine learning systems are becoming increasingly ubiquitous. The main reason for this is that these methods boast remarkable predictive capabilities. However, most of these models remain black boxes, meaning that it is very challenging for humans to follow and understand their intricate inner workings. Consequently, interpretability has suffered under this ever-increasing complexity of machine learning models. Especially with regards to new regulations, such as the General Data Protection Regulation (GDPR), the necessity for plausibility and verifiability of predictions made by these black boxes is indispensable. Driven by the needs of industry and practice, the research community has recognised this interpretability problem and focussed on developing a growing number of so-called explanation methods over the past few years. These methods explain individual predictions made by black box machine learning models and help to recover some of the lost interpretability. With the proliferation of these explanation methods, it is, however, often unclear, which explanation method offers a higher explanation quality, or is generally better-suited for the situation at hand. In this thesis, we thus propose an axiomatic framework, which allows comparing the quality of different explanation methods amongst each other. Through experimental validation, we find that the developed framework is useful to assess the explanation quality of different explanation methods and reach conclusions that are consistent with independent research.

研究动机与目标

• 为解决黑箱机器学习模型中局部解释方法缺乏标准化评估标准的问题。
• 通过一组公理形式化解释方法的理想特性，以确保逻辑一致性和可靠性。
• 提供一种系统性、理论驱动的方法，用于评估个体预测的解释质量。
• 支持在实际机器学习应用中开发和选择稳健、可信的解释技术。
• 弥合理论上的理想特性与实际模型可解释性评估之间的差距。

提出的方法

• 该框架基于一个形式化的公理系统，定义了解释方法所需的一组最小且必要的特性。
• 关键公理包括忠实性、局部准确性与稳定性，以确保解释能准确反映模型在预测点附近的决策行为。
• 通过逻辑与定量验证检查解释方法对公理的符合程度，来评估其解释质量。
• 引入一个结构化的评估流程，分别且组合地测试解释方法对每个公理的满足情况。
• 将该框架应用于现有解释技术（如 LIME、SHAP）以评估其对公理特性的遵循程度。
• 基于形式化逻辑的方法实现了对解释质量的严格、可复现且透明的评估。

实验结果

研究问题

• RQ1 什么样的核心特性是局部解释方法必须满足，才能被视为可靠且有意义？
• RQ2 如何正式定义并验证黑箱模型中个体预测的解释质量？
• RQ3 现有解释方法（如 LIME 和 SHAP）在多大程度上满足所提出的公理化标准？
• RQ4 公理化框架能否作为比较不同解释技术的通用基准？
• RQ5 公理违反如何影响模型解释的可信度与可解释性？

主要发现

• 公理化框架成功识别出现有解释方法中的关键缺陷，例如在输入扰动下行为不一致。
• LIME 等方法被发现违反了稳定性公理，表明其在输入发生微小变化时会产生不可靠的解释。
• SHAP 在忠实性与局部准确性公理方面表现出更强的符合度，表明其在局部解释质量方面具有更高的可靠性。
• 该框架揭示了目前尚无任何方法能完全满足所有公理，凸显了改进解释技术的必要性。
• 公理化评估过程提供了一种透明、可重复且基于理论的解释质量评估方法。
• 本研究确立了公理化符合性是高风险应用场景中可信模型解释的必要条件。