[论文解读] Human-Aligned Calibration for AI-Assisted Decision Making

Nina L. Corvelo Benz, Manuel Gomez-Rodriguez|arXiv (Cornell University)|May 31, 2023

Explainable Artificial Intelligence (XAI)被引用 6

一句话总结

本文提出了一种面向人类对齐的校准方法，用于人工智能辅助决策，其中置信度分数被调整以匹配人类决策者的信任模式。通过确保相对于人类置信度的多重校准，该方法实现了单调的信任行为，相较于标准校准，在四个现实世界任务中均提升了决策质量。

ABSTRACT

Whenever a binary classifier is used to provide decision support, it typically provides both a label prediction and a confidence value. Then, the decision maker is supposed to use the confidence value to calibrate how much to trust the prediction. In this context, it has been often argued that the confidence value should correspond to a well calibrated estimate of the probability that the predicted label matches the ground truth label. However, multiple lines of empirical evidence suggest that decision makers have difficulties at developing a good sense on when to trust a prediction using these confidence values. In this paper, our goal is first to understand why and then investigate how to construct more useful confidence values. We first argue that, for a broad class of utility functions, there exist data distributions for which a rational decision maker is, in general, unlikely to discover the optimal decision policy using the above confidence values -- an optimal decision maker would need to sometimes place more (less) trust on predictions with lower (higher) confidence values. However, we then show that, if the confidence values satisfy a natural alignment property with respect to the decision maker's confidence on her own predictions, there always exists an optimal decision policy under which the level of trust the decision maker would need to place on predictions is monotone on the confidence values, facilitating its discoverability. Further, we show that multicalibration with respect to the decision maker's confidence on her own predictions is a sufficient condition for alignment. Experiments on four different AI-assisted decision making tasks where a classifier provides decision support to real human experts validate our theoretical results and suggest that alignment may lead to better decisions.

研究动机与目标

• 为解决人类决策者在人工智能辅助决策中难以理解标准校准置信度分数的实证挑战。
• 使用结构因果模型（SCM）形式化人工智能辅助决策，以分析信任动态。
• 识别在何种条件下，校准后的置信度值无法支持理性用户的最优决策策略。
• 提出一种新的对齐属性——人类对齐性（human-alignment），以确保单调的信任行为和最优决策。
• 通过真实世界实验验证，人类对齐的置信度相较于标准校准能带来更优的决策。

提出的方法

• 使用结构因果模型（SCM）形式化人工智能辅助决策，以表示人类与人工智能预测之间的交互。
• 将人类对齐性定义为一种属性，即决策者对人工智能预测所赋予的置信度水平在所提供的置信度值上呈单调性。
• 证明相对于人类自身置信度的多重校准是实现人类对齐性的充分条件。
• 提出一种后处理算法，通过 $λ$-离散化和迭代优化，实现 $(\alpha + \lambda)$-多重校准。
• 利用微差隐私和自适应数据分析工具对期望值和概率进行经验估计，以实现实际应用。
• 将该方法应用于四个真实世界人工智能辅助决策任务的数据集，将原始置信度分数转换为 [0,1] 区间，并比较不同决策策略。

实验结果

研究问题

• RQ1 在何种条件下，标准校准置信度值无法支持理性人类用户的最优决策？
• RQ2 置信度分数的何种属性可确保理性决策者能够持续且正确地调节对人工智能预测的信任？
• RQ3 能否利用相对于人类置信度的多重校准来构建与人类决策行为一致的置信度值？
• RQ4 人类对齐的置信度是否能提升真实世界人工智能辅助任务中的决策表现？
• RQ5 当实现对齐时，人类决策者对人工智能预测的信任水平是否随所提供置信度值单调递增？

主要发现

• 对于一大类效用函数，标准校准置信度值可能误导理性决策者，导致无法实现最优决策。
• 人类对齐性确保了最优决策策略的存在，其中信任随置信度单调增加，从而提升了可发现性和可用性。
• 相对于人类自身置信度的多重校准是实现人类对齐性的充分条件。
• 在四个真实世界人工智能辅助决策任务上的实验表明，采用人类对齐置信度的分类器比标准校准置信度的分类器表现出更优的决策性能。
• 结果表明，当实现对齐时，人类决策者的信任水平随分类器置信度单调递增。
• 后处理算法成功地将原始置信度分数转化为满足 $(\alpha + \lambda)$-多重校准的人类对齐值，从而改善了决策结果。