[Paper Review] Designer-User Communication for XAI: An epistemological approach to discuss XAI design

Juliana Jansen Ferreira, Mateus de Souza Monteiro|arXiv (Cornell University)|May 17, 2021

Big Data and Business Intelligence4 citations

TL;DR

This paper proposes an epistemological framework using the Signifying Message concept to operationalize early-stage communication between AI designers, developers, and end-users in XAI (Explainable AI) design. By applying this tool in a healthcare AI case study, the authors demonstrate how shared understanding of explanations can be co-constructed, improving alignment across stakeholders and enabling more human-centered XAI development from the outset.

ABSTRACT

Artificial Intelligence is becoming part of any technology we use nowadays. If the AI informs people's decisions, the explanation about AI's outcomes, results, and behavior becomes a necessary capability. However, the discussion of XAI features with various stakeholders is not a trivial task. Most of the available frameworks and methods for XAI focus on data scientists and ML developers as users. Our research is about XAI for end-users of AI systems. We argue that we need to discuss XAI early in the AI-system design process and with all stakeholders. In this work, we aimed at investigating how to operationalize the discussion about XAI scenarios and opportunities among designers and developers of AI and its end-users. We took the Signifying Message as our conceptual tool to structure and discuss XAI scenarios. We experiment with its use for the discussion of a healthcare AI-System.

Motivation & Objective

• To address the gap in stakeholder-inclusive XAI design by focusing on end-users beyond data scientists.
• To operationalize discussions about XAI scenarios among designers, developers, and end-users during early AI system development.
• To propose a conceptual framework that supports epistemological alignment in XAI explanation design.
• To validate the framework through a practical application in a healthcare AI system context.

Proposed method

• The Signifying Message framework is used as a conceptual tool to structure communication about AI explanations among diverse stakeholders.
• The method involves co-design sessions where designers, developers, and end-users collaboratively interpret and articulate the meaning of AI outputs.
• The framework emphasizes the epistemological dimension—how knowledge about AI behavior is constructed and shared.
• A healthcare AI system was used as a case study to demonstrate the application of Signifying Messages in real-world XAI design scenarios.
• The approach integrates insights from human-computer interaction (HCI) and AI explainability to support iterative, dialogic design processes.
• The process enables stakeholders to negotiate what counts as a 'satisfying' explanation based on contextual and domain-specific needs.

Experimental results

Research questions

• RQ1 How can designers and end-users collaboratively discuss and shape the form and content of AI explanations in early development stages?
• RQ2 What conceptual tools can support epistemological alignment in XAI design across diverse stakeholders?
• RQ3 How does the Signifying Message framework facilitate communication about AI explanations in a healthcare context?
• RQ4 In what ways does involving end-users early improve the relevance and usability of XAI features?
• RQ5 What are the practical challenges and opportunities in using signifying messages to co-define explanation needs?

Key findings

• The Signifying Message framework successfully enabled structured, epistemologically grounded dialogue between designers, developers, and end-users in a healthcare AI context.
• Stakeholders were able to co-define what constitutes a meaningful explanation by focusing on the signification of AI outputs within specific clinical contexts.
• The method revealed that end-users' understanding of explanations is deeply tied to their domain knowledge and decision-making needs.
• Designers and developers gained new insights into the practical and ethical dimensions of explanation that were previously overlooked in technical XAI approaches.
• The framework supported the identification of context-specific explanation requirements that were not apparent through technical analysis alone.
• The study demonstrated that early, inclusive communication about explanations leads to more usable and trustworthy AI systems.