[论文解读] Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making

Shuai Ma, Qiaoyi Chen+5 more|arXiv (Cornell University)|Mar 25, 2024

Ethics and Social Impacts of AI被引用 7

一句话总结

本论文提出一个人-AI 深思框架和一个结合领域模型与大语言模型的深思AI，以实现决策中的维度级别的人机讨论，在研究生招生任务中进行演示。

ABSTRACT

In AI-assisted decision-making, humans often passively review AI's suggestion and decide whether to accept or reject it as a whole. In such a paradigm, humans are found to rarely trigger analytical thinking and face difficulties in communicating the nuances of conflicting opinions to the AI when disagreements occur. To tackle this challenge, we propose Human-AI Deliberation, a novel framework to promote human reflection and discussion on conflicting human-AI opinions in decision-making. Based on theories in human deliberation, this framework engages humans and AI in dimension-level opinion elicitation, deliberative discussion, and decision updates. To empower AI with deliberative capabilities, we designed Deliberative AI, which leverages large language models (LLMs) as a bridge between humans and domain-specific models to enable flexible conversational interactions and faithful information provision. An exploratory evaluation on a graduate admissions task shows that Deliberative AI outperforms conventional explainable AI (XAI) assistants in improving humans' appropriate reliance and task performance. Based on a mixed-methods analysis of participant behavior, perception, user experience, and open-ended feedback, we draw implications for future AI-assisted decision tool design.

研究动机与目标

• 通过在冲突的人类与AI意见上进行深思/协商来解决人类在决策中对AI过度依赖或不足依赖的局限性。
• 开发一个使用证据权重（Weight of Evidence）来引出、对齐、讨论并更新来自人类和AI的意见的框架。
• 设计并实现一个深思AI，将领域特定模型与LLMs整合，以实现互动式思考和忠实信息提供。
• 在研究生招生情景中以实证方式探索深思如何影响决策表现、对AI的依赖、用户感知与体验。

提出的方法

• 提出一个基于证据权重的四组件框架（引出、对齐、讨论、更新），用于维度级别的深思。
• 引入将领域特定模型（DS 模型）与LLMs融合的深思AI架构，横跨三个层次：通信、控制和知识。
• 定义设计考量（参与平等、论证合理性、建设性更新、互动性、尊重），并实现一台符合这些标准的深思AI。
• 在研究生招生任务中使用合成的申请者档案来实现该框架，以将深思AI与传统XAI和人类基线进行比较。
• 收集混合方法数据，包括任务表现、对AI依赖的感知、用户体验、对话日志和开放式反馈。

Figure 1. An illustration of Human-AI Deliberation . (A) In traditional AI-assisted decision-making, when humans disagree with AI’s suggestions (and only find parts of AI’s reasons applaudable), it is difficult for humans to decide whether and how much to adopt AI’s suggestion. (B) In our proposed H

实验结果

研究问题

• RQ1 RQ1：人机深思在具有挑战性的任务案例中如何影响任务表现和对AI的恰当依赖程度（对人和AI均有影响）。
• RQ2 RQ2：该框架如何影响人类对AI的感知和用户体验。
• RQ3 RQ3：在人机交互中，人类在深思过程中的参与方式如何？
• RQ4 RQ4：用户如何看待人机深思的有效性，以及提出了哪些改进建议。

主要发现

• 深思AI相较于传统XAI助手，在提高决策准确性方面显示出潜力。
• 深思通过更认真地考虑AI推理来促进对AI的恰当依赖。
• 参与者的行为、感知与反馈为界面设计和AI辅助决策工具的未来改进提供了见解。
• 研究生招生任务展示了将深思整合到AI辅助决策中的可行性和价值。

Figure 2. The framework for Human-AI Deliberation . (A) Illustrates the Weight of Evidence (WoE) concept in decision-making, showcasing how decision-makers assess evidence across dimensions to shape opinions and arrive at a final decision. (B) Presents the Architecture for Human-AI Deliberation , wi