[Paper Review] Why Don't You Do Something About It? Outlining Connections between AI Explanations and User Actions

Gennie Mansi, Mark Riedl|arXiv (Cornell University)|May 10, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes a framework linking explainable AI (XAI) explanations to actionable user responses, identifying 10 information categories in explanations and their associated user actions. By mapping information types to concrete actions, the study reorients XAI evaluation toward real-world user agency, addressing the gap in connecting explanation content to tangible user behavior in socio-technical systems.

ABSTRACT

A core assumption of explainable AI systems is that explanations change what users know, thereby enabling them to act within their complex socio-technical environments. Despite the centrality of action, explanations are often organized and evaluated based on technical aspects. Prior work varies widely in the connections it traces between information provided in explanations and resulting user actions. An important first step in centering action in evaluations is understanding what the XAI community collectively recognizes as the range of information that explanations can present and what actions are associated with them. In this paper, we present our framework, which maps prior work on information presented in explanations and user action, and we discuss the gaps we uncovered about the information presented to users.

Motivation & Objective

• To address the lack of consensus on what information should be included in XAI explanations and how it connects to user actions.
• To recenter evaluation of XAI systems around user action, rather than solely technical metrics.
• To identify and categorize information types in explanations that enable users to take meaningful actions.
• To provide a structured, evidence-based framework linking explanation content to actionable outcomes.
• To support human-centered design of XAI systems by aligning explanation content with user agency and socio-technical context.

Proposed method

• The authors conducted a systematic literature review of 30 survey papers on XAI, selecting 11 that focused on explanation content and user actions.
• They synthesized information types from existing work, particularly drawing from the ICO & Turing Institute’s taxonomy, and iteratively refined 10 thematic categories of explanation information.
• For each information category, they explicitly identified and listed corresponding user actions that could logically follow from the explanation.
• The framework was developed through iterative classification and redefinition, ensuring thematic coherence and alignment with user action goals.
• The final framework organizes explanation information into three clusters: Model Exposure (how the model makes decisions), Model Accountability (who built it and why), and Model Context (how it fits into user context and external systems).
• The framework is presented as a visual mapping tool to guide designers and evaluators in aligning explanation content with actionable outcomes.

Experimental results

Research questions

• RQ1 What kinds of information do XAI explanations typically present, and how do they relate to user actions?
• RQ2 What are the key gaps in the current literature regarding the connection between explanation content and user agency?
• RQ3 How can explanation content be systematically categorized to support actionable user responses in real-world socio-technical systems?
• RQ4 What are the most salient user actions that explanations should enable, and how can they be linked to specific information types?
• RQ5 How can XAI evaluation shift from technical metrics to actionability as a core heuristic?

Key findings

• The study identified 10 distinct categories of information that explanations can convey, grouped into three clusters: Model Exposure, Model Accountability, and Model Context.
• For each information category, the framework explicitly lists actionable responses users can take, such as correcting input data, seeking alternative decisions, or challenging model outcomes.
• The analysis revealed a significant gap in the literature: while many papers describe explanation types, few explicitly link them to user actions or actionability.
• The framework demonstrates that explanations are often evaluated based on technical accuracy or interpretability, not on their ability to enable meaningful user intervention.
• The study shows that aligning explanation content with user actions can enhance trust, transparency, and perceived agency in human-AI interactions.
• The framework provides a structured, reusable foundation for evaluating XAI systems based on their capacity to support user-driven decision-making and system-level interventions.