[论文解读] `Why not give this work to them?' Explaining AI-Moderated Task-Allocation Outcomes using Negotiation Trees

Zahra Zahedi, Sailik Sengupta+1 more|arXiv (Cornell University)|Feb 5, 2020

Explainable Artificial Intelligence (XAI)参考文献 22被引用 4

一句话总结

本文提出了一种人工智能任务分配器（AITA），通过生成谈判树来解释为何替代性（反事实）的分配方案会表现不佳，从而在人类-多智能体团队中实现可解释且公平的任务分配。研究发现，人类能够理解并被这些解释所说服，尤其是在低估他人成本时，解释虽更长但更具信息量。

ABSTRACT

The problem of multi-agent task allocation arises in a variety of scenarios involving human teams. In many such settings, human teammates may act with selfish motives and try to minimize their cost metrics. In the absence of (1) complete knowledge about the reward of other agents and (2) the team's overall cost associated with a particular allocation outcome, distributed algorithms can only arrive at sub-optimal solutions within a reasonable amount of time. To address these challenges, we introduce the notion of an AI Task Allocator (AITA) that, with complete knowledge, comes up with fair allocations that strike a balance between the individual human costs and the team's performance cost. To ensure that AITA is explicable to the humans, we allow each human agent to question AITA's proposed allocation with counterfactual allocations. In response, we design AITA to provide a replay negotiation tree that acts as an explanation showing why the counterfactual allocation, with the correct costs, will eventually result in a sub-optimal allocation. This explanation also updates a human's incomplete knowledge about their teammate's and the team's actual costs. We then investigate whether humans are (1) able to understand the explanations provided and (2) convinced by it using human factor studies. Finally, we show the effect of various kinds of incompleteness on the length of explanations. We conclude that underestimation of other's costs often leads to the need for explanations and in turn, longer explanations on average.

研究动机与目标

• 解决因对个体和团队成本认知不全而导致的人类团队任务分配次优的问题。
• 设计一种具备完整知识的AI任务分配器（AITA），实现个体与团队成本之间的公平、均衡分配。
• 使AITA能够通过重放的谈判树解释其分配决策，展示反事实分配为何会失败。
• 评估人类是否能在人因实验中理解并被这些解释所说服。
• 分析不同类型的成本认知不全对解释长度与清晰度的影响。

提出的方法

• AITA利用对个体与团队成本的完整知识，计算出在个体努力与团队绩效之间达到平衡的公平任务分配。
• 当人类智能体质疑分配结果时，AITA接受一个反事实分配方案，并使用重放的谈判树模拟其协商过程。
• 谈判树通过追踪成本传播与决策路径，解释为何反事实分配最终会导致次优结果。
• 通过揭示人类此前缺乏的准确成本信息，动态更新其对任务成本的理解。
• 通过人因实验评估解释的理解程度与说服力。
• 测量解释长度，并与不同类型的知识不全（尤其是对他人成本的低估）进行相关性分析。

实验结果

研究问题

• RQ1 人类能否理解AITA为AI中介任务分配生成的谈判树解释？
• RQ2 当人类质疑AI的分配提案时，是否会被这些解释所说服？
• RQ3 人类知识的不完全程度（尤其是对他人成本的低估）如何影响解释长度？
• RQ4 该解释机制是否成功更新了人类智能体对其队友及团队整体实际成本的认知？
• RQ5 成本认知不全的类型与所需解释复杂度之间存在何种关系？

主要发现

• 人类能够理解AITA提供的谈判树解释，表明该方法支持有效的跨人类-AI沟通。
• 人类通常会被解释所说服，尤其是在反事实分配会导致明显次优结果时。
• 对他人成本的低估是导致解释更长的主要原因，因为它在谈判树中触发了更复杂的推理路径。
• 解释机制成功更新了人类智能体对其队友及团队整体成本的认知，减少了知识差距。
• 当智能体低估他人成本时，解释长度显著增加，证实此类认知不全需要更详细的说明。
• 该系统表明，可解释AI可通过使AI决策透明且可辩护，从而提升协作任务分配中的信任与接受度。