[Paper Review] Stakeholders in Explainable AI

Alun Preece, Daniel Harborne+3 more|ORCA Online Research @Cardiff (Cardiff University)|Sep 29, 2018

Adversarial Robustness in Machine LearningComputer Science30 references43 citations

TL;DR

The paper argues that explainable AI involves distinct stakeholder communities with different intents and requirements, and proposes a layered, stakeholder-aware view of explanations bridging verification and validation.

ABSTRACT

There is general consensus that it is important for artificial intelligence (AI) and machine learning systems to be explainable and/or interpretable. However, there is no general consensus over what is meant by 'explainable' and 'interpretable'. In this paper, we argue that this lack of consensus is due to there being several distinct stakeholder communities. We note that, while the concerns of the individual communities are broadly compatible, they are not identical, which gives rise to different intents and requirements for explainability/interpretability. We use the software engineering distinction between validation and verification, and the epistemological distinctions between knowns/unknowns, to tease apart the concerns of the stakeholder communities and highlight the areas where their foci overlap or diverge. It is not the purpose of the authors of this paper to 'take sides' - we count ourselves as members, to varying degrees, of multiple communities - but rather to help disambiguate what stakeholders mean when they ask 'Why?' of an AI.

Motivation & Objective

• Identify distinct AI explainability stakeholder communities and their motivations.
• Disentangle how developers, theorists, ethicists, and users differ in their explainability needs.
• Relate explainability to software engineering concepts of verification and validation.
• Propose a layered, multi-modal explanation model to satisfy multiple stakeholders.
• Highlight the importance of user-centered explanations to prevent an AI Winter.

Proposed method

• Classify explainability stakeholders into four communities: developers, theorists, ethicists, and users.
• Use software engineering concepts (verification vs. validation) to map explanation types to stakeholder needs.
• Differentiate between transparency-based (verification) and post-hoc (validation) explanations.
• Discuss knowns/unknowns epistemology to analyze how explanations address different information gaps.
• Propose a layered explanation object (traceability, justification, assurance) tailored to stakeholder requirements.
• Provide example wildlife monitoring scenario to illustrate layered explanations.

Experimental results

Research questions

• RQ1 What are the distinct stakeholder communities of explainable AI and their primary motivations?
• RQ2 How do verification and validation concepts map onto explanation types across stakeholder groups?
• RQ3 What epistemological categories (knowns/unknowns) illuminate how explanations should be designed for different recipients?
• RQ4 Can a layered, multi-modal explanation framework satisfy multiple stakeholders without overwhelming any single group?
• RQ5 What role do end-users play relative to other stakeholders in the explainable AI discourse?

Key findings

• There are four stakeholder communities for explainable AI: developers, theorists, ethicists, and users, each with distinct aims for explanations.
• Developers/theorists focus more on verification and transparency-based explanations, while users/ethicists emphasize validation and post-hoc explanations.
• A layered explanation approach (traceability, justification, assurance) can satisfy multiple stakeholders by organizing information appropriately.
• Transparency-based explanations are often unstable or hard to interpret for non-developer/ethicist recipients, requiring careful presentation.
• Post-hoc explanations, including example-based and textual explanations, are particularly suitable for users and ethicists but must be clearly labeled as approximations.
• End-user perspectives are crucial and often underrepresented but essential to avoid future AI adoption setbacks.