[論文レビュー] The Who in XAI: How AI Background Shapes Perceptions of AI Explanations

Upol Ehsan, Samir Passi+5 more|arXiv (Cornell University)|Jul 28, 2021

Explainable Artificial Intelligence (XAI)被引用数 29

ひとこと要約

この研究は、AI-オタクと非AI参加者が三種類のAI説明をどのように知覚するかを比較し、グループ固有の信頼、解釈、および人間らしさの差異を明らかにし、デザイン上の示唆を得る。

ABSTRACT

Explainability of AI systems is critical for users to take informed actions. Understanding "who" opens the black-box of AI is just as important as opening it. We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. Quantitatively, we share user perceptions along five dimensions. Qualitatively, we describe how AI background can influence interpretations, elucidating the differences through lenses of appropriation and cognitive heuristics. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design. Carrying critical implications for the field of XAI, our findings showcase how AI generated explanations can have negative consequences despite best intentions and how that could lead to harmful manipulation of trust. We propose design interventions to mitigate them.

研究の動機と目的

• Quantify user preferences for three AI explanation types along five perception dimensions.
• Qualitatively examine how AI background shapes interpretation and meaning-making of explanations.
• Explain why group differences arise using heuristics and appropriation as analytic lenses.
• Identify potential negative consequences (e.g., over-trust) and propose design interventions.
• Advance a pluralistic, human-centered discourse in XAI by highlighting creator-consumer gaps.

提案手法

• Conduct a within-subjects experiment with two groups (AI background vs. non-AI background)
• Expose participants to three AI agents delivering different explanation styles: rationale generation (RG), action-declaring (AD), and numerical-reasoning (NR)
• Use a reinforcement-learning (tabular Q-learning) agent in a sequential navigation task
• Measure perceptions across five dimensions: understandability, confidence, intelligence, friendliness, and second chance
• Supplement quantitative data with qualitative analyses of open-ended responses to interpret group differences
• Ground interpretations in theoretical lenses of heuristics and appropriation.

実験結果

リサーチクエスチョン

• RQ1 RQ1: Quantitatively, what are the effects of different types of explanations on how people with or without an AI background perceive AI agents?
• RQ2 RQ2: Qualitatively, how and why do differences in AI background result in different perceptions of explanations?

主な発見

• Both groups exhibited unwarranted faith in numerical representations, though for different reasons and to different extents.
• Each group ascribed explanatory value beyond the intended use of the explanations, suggesting divergent interpretation goals.
• Even when appreciating humanlike qualities, the groups differed in what counts as humanlike explanations and why they find them persuasive.
• The findings highlight risks such as potential manipulation of user trust and over-trust in XAI systems.
• The study discusses design interventions to mitigate these risks and emphasizes a pluralistic, human-centered approach to XAI design.