[Paper Review] Machine Learning Explainability for External Stakeholders

Umang Bhatt, McKane Andrus+2 more|arXiv (Cornell University)|Jul 10, 2020

Explainable Artificial Intelligence (XAI)44 references40 citations

TL;DR

A workshop-based study exploring how to make explainable ML useful for external stakeholders (end-users, regulators, domain experts) and outlining open challenges for deploying explanations at scale.

ABSTRACT

As machine learning is increasingly deployed in high-stakes contexts affecting people's livelihoods, there have been growing calls to open the black box and to make machine learning algorithms more explainable. Providing useful explanations requires careful consideration of the needs of stakeholders, including end-users, regulators, and domain experts. Despite this need, little work has been done to facilitate inter-stakeholder conversation around explainable machine learning. To help address this gap, we conducted a closed-door, day-long workshop between academics, industry experts, legal scholars, and policymakers to develop a shared language around explainability and to understand the current shortcomings of and potential solutions for deploying explainable machine learning in service of transparency goals. We also asked participants to share case studies in deploying explainable machine learning at scale. In this paper, we provide a short summary of various case studies of explainable machine learning, lessons from those studies, and discuss open challenges.

Motivation & Objective

• Clarify why explainability should reach external stakeholders beyond internal model developers.
• Assess current shortcomings in deploying explanations to end users, regulators, and domain experts.
• Summarize domain-specific use cases and stakeholder needs to align explainability with transparency goals.
• Identify open challenges and potential solutions for deploying explainable ML at scale.
• Propose interdisciplinary engagement to improve the development and deployment of explanations.

Proposed method

• Conducted a closed-door, day-long workshop with 33 participants from academia, industry, policy, and legal backgrounds.
• Facilitated group discussions across five expert clusters to align definitions of explainability.
• Analyzed domain-specific use cases in finance, healthcare, media, and social services.
• Synthesized participant-defined explanations and lessons learned into a summary of case studies and open challenges.
• Highlighted need for broader community engagement and education around explainability.

Experimental results

Research questions

• RQ1 What definitions and conceptions of explainability emerge from interdisciplinary discussions?
• RQ2 What are the key challenges and potential solutions for deploying explainable ML for external stakeholders at scale?
• RQ3 How should explanations be evaluated, and what roles do context and stakeholder needs play?
• RQ4 How can explanations be designed to account for stakeholder diversity, data use, and privacy concerns?
• RQ5 What role do uncertainty, interactivity, and time-evolving behavior play in explainable ML deployments?

Key findings

• Explainability currently remains internal-focused and is not widely deployed to external stakeholders.
• Effective deployment requires context-aware explanations tailored to specific stakeholders and use cases.
• Evaluation of explanations is unclear and benefits from interdisciplinary human-centered methods.
• Engaging affected communities and educating stakeholders are essential for practical adoption.
• Uncertainty should be considered alongside explanations, and explanations should be interactive and adaptable to stakeholder feedback.
• Explanations should anticipate changes in model behavior and user interaction over time to avoid degradation of usefulness.