[论文解读] Comparing Explanation Methods for Traditional Machine Learning Models Part 2: Quantifying Model Explainability Faithfulness and Improvements with Dimensionality Reduction

Montgomery L. Flora, Corey K. Potvin+2 more|arXiv (Cornell University)|Nov 18, 2022

Explainable Artificial Intelligence (XAI)被引用 11

一句话总结

本研究评估了用于大气科学的机器学习模型中特征重要性排序方法的忠实度，表明在减少相关特征后，置换重要性（permutation importance）成为最忠实的方法。降维显著提升了可解释性的忠实度，并将排序不确定性降低了2至10倍，为气象学应用中更可靠的模型解释提供了实用路径。

ABSTRACT

Machine learning (ML) models are becoming increasingly common in the atmospheric science community with a wide range of applications. To enable users to understand what an ML model has learned, ML explainability has become a field of active research. In Part I of this two-part study, we described several explainability methods and demonstrated that feature rankings from different methods can substantially disagree with each other. It is unclear, though, whether the disagreement is overinflated due to some methods being less faithful in assigning importance. Herein, "faithfulness" or "fidelity" refer to the correspondence between the assigned feature importance and the contribution of the feature to model performance. In the present study, we evaluate the faithfulness of feature ranking methods using multiple methods. Given the sensitivity of explanation methods to feature correlations, we also quantify how much explainability faithfulness improves after correlated features are limited. Before dimensionality reduction, the feature relevance methods [e.g., SHAP, LIME, ALE variance, and logistic regression (LR) coefficients] were generally more faithful than the permutation importance methods due to the negative impact of correlated features. Once correlated features were reduced, traditional permutation importance became the most faithful method. In addition, the ranking uncertainty (i.e., the spread in rank assigned to a feature by the different ranking methods) was reduced by a factor of 2-10, and excluding less faithful feature ranking methods reduces it further. This study is one of the first to quantify the improvement in explainability from limiting correlated features and knowing the relative fidelity of different explainability methods.

研究动机与目标

• 评估用于强对流天气和路面温度预测的机器学习模型中，各种特征排序方法的忠实度。
• 研究特征相关性如何加剧不同可解释性方法之间的分歧，并探讨降维是否能缓解此问题。
• 识别在具有高维、相关特征的真实地球科学数据集中，哪些解释方法最为忠实。
• 通过排除低忠实度方法并应用特征降维技术，降低排序不确定性。
• 为从业者提供在气象机器学习应用中选择可靠可解释性方法的实际指导。

提出的方法

• 本研究使用两个大气科学数据集：来自WoFS输出的强对流天气预测和路面温度低于冰点的预测。
• 评估了多种特征排序方法，包括SHAP、LIME、ALE方差、逻辑回归系数以及各种置换重要性变体（如前向/后向传播、单次/多次遍历）。
• 忠实度采用Covert等人（2020）提出的方法进行量化，该方法通过测量特征重要性与模型性能变化之间的一致性，无需真实标签即可估计忠实度。
• 通过降维技术减少相关特征，以评估其对解释忠实度和排序不确定性的影响。
• 排序不确定性通过不同方法间特征排名的差异范围来计算，其减少程度在排除低忠实度方法后被量化。
• 分析在逻辑回归和随机森林模型上进行，模型在原始和降维后的特征集上训练，超参数通过贝叶斯优化进行调优。

实验结果

研究问题

• RQ1 不同特征排序方法在多大程度上忠实于实际模型性能贡献？
• RQ2 特征相关性在多大程度上加剧了不同解释方法之间的分歧？
• RQ3 降维如何影响特征排序方法的忠实度？
• RQ4 排除低忠实度解释方法是否能降低排序不确定性？
• RQ5 在去除相关特征后，哪种特征排序方法变得最忠实？

主要发现

• 在降维之前，由于相关特征的负面影响，SHAP、LIME、ALE方差和逻辑回归系数通常比置换重要性更忠实。
• 在减少相关特征后，传统置换重要性成为最忠实的方法，优于其他方法。
• 排序不确定性——以不同方法间特征排名的差异范围衡量——在降维后降低了2至10倍。
• 排除较不忠实的特征排序方法进一步降低了排序不确定性，表明方法选择能提升可解释性的可靠性。
• 随着维度增加，不同方法间的忠实度差异减小，表明高维数据集需要对可解释性方法进行更细致的评估。
• 特征排序方法可作为稳健的特征选择工具，提供一种计算效率更高的传统方法替代方案。