[论文解读] Can predictive models be used for causal inference?

Maximilian Pichler, Florian Härtig|arXiv (Cornell University)|Jun 18, 2023

Explainable Artificial Intelligence (XAI)被引用 5

一句话总结

本文表明，当特征选择遵循Pearl的后门准则时，预测性机器学习（ML）和深度学习（DL）模型可被有效重用于因果推断。通过将模型学习限制在因果特征上，深度神经网络即使在特征共线性情况下也能实现近乎无偏的因果效应估计，且泛化性能显著提升——这挑战了长期以来关于预测与解释之间存在根本性权衡的信念。

ABSTRACT

Supervised machine learning (ML) and deep learning (DL) algorithms excel at predictive tasks, but it is commonly assumed that they often do so by exploiting non-causal correlations, which may limit both interpretability and generalizability. Here, we show that this trade-off between explanation and prediction is not as deep and fundamental as expected. Whereas ML and DL algorithms will indeed tend to use non-causal features for prediction when fed indiscriminately with all data, it is possible to constrain the learning process of any ML and DL algorithm by selecting features according to Pearl's backdoor adjustment criterion. In such a situation, some algorithms, in particular deep neural networks, can provide near unbiased effect estimates under feature collinearity. Remaining biases are explained by the specific algorithmic structures as well as hyperparameter choice. Consequently, optimal hyperparameter settings are different when tuned for prediction or inference, confirming the general expectation of a trade-off between prediction and explanation. However, the effect of this trade-off is small compared to the effect of a causally constrained feature selection. Thus, once the causal relationship between the features is accounted for, the difference between prediction and explanation may be much smaller than commonly assumed. We also show that such causally constrained models generalize better to new data with altered collinearity structures, suggesting generalization failure may often be due to a lack of causal learning. Our results not only provide a perspective for using ML for inference of (causal) effects but also help to improve the generalizability of fitted ML and DL models to new data.

研究动机与目标

• 探究尽管存在关于非因果相关性的常见假设，预测性ML和DL模型是否仍可用于因果推断。
• 评估基于因果信息的特征选择对因果效应估计中模型偏差和泛化能力的影响。
• 比较为预测与推断分别进行的超参数调优，识别最优设置的差异。
• 在改变共线性结构的条件下评估模型性能，检验其在训练数据之外的泛化能力。
• 挑战机器学习中预测准确性与因果可解释性之间存在根本性权衡的普遍看法。

提出的方法

• 应用Pearl的后门准则选择可阻断后门混淆路径的特征，确保因果识别。
• 在因果选择的特征上训练标准的ML和深度学习模型（如前馈神经网络），以估计因果效应。
• 使用反事实预测和效应估计来评估所学模型的偏差和精度。
• 在不同水平的特征共线性下比较模型性能，以评估其鲁棒性和泛化能力。
• 分别针对预测和推断目标进行超参数调优，以隔离优化目标的影响。
• 开展模拟研究，评估在多样化数据生成过程下的偏差、方差和泛化能力。

实验结果

研究问题

• RQ1 当在因果选择的特征上进行训练时，预测性ML模型能否产生无偏的因果效应估计？
• RQ2 基于后门准则的特征选择如何影响因果推断中模型的偏差和泛化能力？
• RQ3 当模型在因果特征上进行训练时，预测性能与因果可解释性之间是否存在显著的权衡？
• RQ4 在为预测与因果推断分别优化时，超参数选择有何不同？
• RQ5 在数据分布发生改变的情况下，因果特征选择在多大程度上提升了模型的泛化能力？

主要发现

• 通过因果约束的特征选择，深度神经网络即使在高特征共线性下也能实现近乎无偏的因果效应估计。
• 模型估计中的偏差主要由算法结构和超参数选择决定，而非非因果特征的存在，前提是选择了适当的特征。
• 在因果特征选择下训练的模型在具有不同共线性结构的新数据上表现出显著更好的泛化能力，表明泛化失败通常源于缺乏因果学习。
• 为推断优化的超参数调优需要与为预测优化不同的设置，证实了存在权衡——但这种权衡相比因果特征选择的影响微不足道。
• 当在特征选择中适当地考虑因果结构时，预测性能与因果可解释性之间的差距远小于普遍认为的那样。
• 因果特征选择减少了对虚假相关性的依赖，从而增强了机器学习模型在因果情境下的可解释性和鲁棒性。