[论文解读] PermuteAttack: Counterfactual Explanation of Machine Learning Credit Scorecards

Masoud Hashemi, Ali Fathi|arXiv (Cornell University)|Aug 24, 2020

Adversarial Robustness in Machine Learning参考文献 23被引用 30

一句话总结

简介 PermuteAttack，一种无梯度、基于遗传算法的方法，用于为表格信用评分模型生成现实的对反事实( counterfactuals )，实现实例级解释和模型评估。

ABSTRACT

This paper is a note on new directions and methodologies for validation and explanation of Machine Learning (ML) models employed for retail credit scoring in finance. Our proposed framework draws motivation from the field of Artificial Intelligence (AI) security and adversarial ML where the need for certifying the performance of the ML algorithms in the face of their overwhelming complexity poses a need for rethinking the traditional notions of model architecture selection, sensitivity analysis and stress testing. Our point of view is that the phenomenon of adversarial perturbations when detached from the AI security domain, has purely algorithmic roots and fall within the scope of model risk assessment. We propose a model criticism and explanation framework based on adversarially generated counterfactual examples for tabular data. A counterfactual example to a given instance in this context is defined as a synthetically generated data point sampled from the estimated data distribution which is treated differently by a model. The counterfactual examples can be used to provide a black-box instance-level explanation of the model behaviour as well as studying the regions in the input space where the model performance deteriorates. Adversarial example generating algorithms are extensively studied in the image and natural language processing (NLP) domains. However, most financial data come in tabular format and naive application of the existing techniques on this class of datasets generates unrealistic samples. In this paper, we propose a counterfactual example generation method capable of handling tabular data including discrete and categorical variables. Our proposed algorithm uses a gradient-free optimization based on genetic algorithms and therefore is applicable to any classification model.

研究动机与目标

• 推动金融领域基于机器学习的零售信用评分的模型验证与解释。
• 提出一种面向表格数据的黑箱对反事实生成框架。
• 通过建模高阶交互，确保对反事实具有现实性且在分布内。
• 提供可视化和统计工具来分析特征交互和模型敏感性。

提出的方法

• 使用一个无梯度的遗传算法（PermuteAttack）通过置换从训练数据中选取的特征值来搜索对反事实。
• 定义一个基于接近性的适应度函数，偏好最小扰动和目标类别的变化。
• 通过置换而非微小的加性噪声来扰动特征，以保持表格数据的有效性，包括离散/分类变量。
• 为置换应用一热编码处理并进行有序转换，然后在评估时重新转换。
• 通过离散化和吉布斯采样引入高阶统计，以从现实的条件分布中采样。
• 报告对反事实实例并突出显示被改变的特征，以解释模型决策。

实验结果

研究问题

• RQ1 是否可以在没有模型梯度的情况下为表格信用评分生成对反事实解释？
• RQ2 基于置换的对反事实是否遵循数据分布，包括离散和分类特征？
• RQ3 翻转预测所需的典型特征集合大小是多少，以及 SHAP 标注的重要特征如何与扰动频率对齐？
• RQ4 对反事实解释如何与特征交互影响模型决策相关？
• RQ5 通过分析敏感属性的扰动，该方法是否能揭示潜在的公平性或歧视信号？

主要发现

• 在测试的超参数下，PermuteAttack 针对德国信贷数据生成的对反事实在将预测翻转方面实现了 100% 的成功。
• 平均需要更改 2.35 个特征以翻转预测。
• 最受扰动的特征与 SHAP 特征重要性高度一致，尽管在不同特征之间顺序可能有所不同。
• 一个共变更图显示特征如何共同被扰动，指示重要特征之间存在强烈交互，如 property、credit_history、account_check_status。
• 将扰动限于敏感属性（age、personal_status_sex、foreign_worker）时，成功率为 26.25%，表明潜在的公平性考虑。
• 移除前列的 SHAP 特征会使改变结果所需的平均特征数量增加到 3.7，强调翻转需要依赖多个属性。
• 吉布斯采样提升对反事实的现实性；训练以区分真实样本与对反事实样本的判别器在使用吉布斯采样时检测到更多对反事实（49% 成功）比未使用时（12%）。