[Paper Review] Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach

Upol Ehsan, Mark Riedl|arXiv (Cornell University)|Feb 3, 2020

Ethics and Social Impacts of AI4 citations

TL;DR

This paper introduces Human-centered Explainable AI (HCXAI), a reflective sociotechnical approach that centers human values, interpersonal dynamics, and social context in AI explanation design. By integrating Critical Technical Practice with HCI strategies like participatory and value-sensitive design, HCXAI enables co-evolution of technical explanation systems and human factor insights, as demonstrated in a rationale generation case study for non-expert users.

ABSTRACT

Explanations--a form of post-hoc interpretability--play an instrumental role in making systems accessible as AI continues to proliferate complex and sensitive sociotechnical systems. In this paper, we introduce Human-centered Explainable AI (HCXAI) as an approach that puts the human at the center of technology design. It develops a holistic understanding of "who" the human is by considering the interplay of values, interpersonal dynamics, and the socially situated nature of AI systems. In particular, we advocate for a reflective sociotechnical approach. We illustrate HCXAI through a case study of an explanation system for non-technical end-users that shows how technical advancements and the understanding of human factors co-evolve. Building on the case study, we lay out open research questions pertaining to further refining our understanding of "who" the human is and extending beyond 1-to-1 human-computer interactions. Finally, we propose that a reflective HCXAI paradigm-mediated through the perspective of Critical Technical Practice and supplemented with strategies from HCI, such as value-sensitive design and participatory design--not only helps us understand our intellectual blind spots, but it can also open up new design and research spaces.

Motivation & Objective

• Address the lack of holistic human-centered design in Explainable AI (XAI) by re-centering the human in AI system development.
• Recognize that 'explainable to whom?' is central to XAI, requiring deep understanding of users' values, social roles, and interpersonal dynamics.
• Advance beyond technical interpretability by embedding sociotechnical reflection into AI design to uncover implicit biases and epistemological blind spots.
• Propose a reflective HCXAI paradigm that combines Critical Technical Practice with participatory and value-sensitive design to foster ethical, democratic, and context-aware AI systems.
• Call for a shift from user acceptance to critical reflection in XAI evaluation, emphasizing skepticism and value alignment over passive trust.

Proposed method

• Adopt a reflective sociotechnical approach that integrates technical development of explanation systems with ongoing human factor analysis.
• Implement rationale generation—natural language explanations simulating human reasoning—to make AI decisions accessible to non-expert users.
• Use participatory design (PD) to involve end-users and stakeholders in co-creating explanation systems, ensuring democratic power dynamics.
• Apply value-sensitive design (VSD) with tools like Envisioning Cards to surface value tensions among stakeholders (e.g., radiologists, administrators, insurers).
• Employ Critical Technical Practice (CTP) to interrogate dominant metaphors and assumptions in XAI, enabling critical reflection on design choices.
• Co-evolve technical systems and human understanding through iterative case studies, where insights from user perception inform system refinement and vice versa.

Experimental results

Research questions

• RQ1 How can we holistically understand 'who' the human is in AI systems, beyond technical roles, to inform explanation design?
• RQ2 What design strategies enable the co-evolution of technical explanation systems and human-centered insights in XAI?
• RQ3 How can we evaluate explanations not just for acceptance, but for fostering critical reflection and skepticism?
• RQ4 What role do values, power dynamics, and social context play in shaping effective and ethical AI explanations?
• RQ5 How can participatory and value-sensitive design practices be operationalized within XAI to ensure stakeholder alignment and ethical transparency?

Key findings

• The co-evolution of technical explanation systems and human factor understanding leads to richer, more context-aware explanations that reflect real user needs.
• Rationale generation effectively communicates AI decisions to non-expert users, improving perceived transparency and trustworthiness.
• Reflective practices such as CTP help uncover implicit values and biases in AI design, enabling more ethical and accountable systems.
• Participatory design fosters democratic engagement and reduces power imbalances between designers and end-users in XAI development.
• Value-sensitive design with Envisioning Cards reveals tensions between stakeholders (e.g., radiologists vs. insurers), highlighting the need for value alignment in XAI systems.
• Evaluating for reflection rather than acceptance reveals deeper challenges in XAI, such as prioritizing critical thinking over passive trust in high-stakes domains.