[Paper Review] One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques

Vijay Arya, Rachel Bellamy+18 more|arXiv (Cornell University)|Sep 6, 2019

Explainable Artificial Intelligence (XAI)46 references263 citations

TL;DR

The paper presents AI Explainability 360 (AIX360), an open-source toolkit with eight explainability methods and two evaluation metrics, plus a practical taxonomy to match explanations to diverse stakeholders.

ABSTRACT

As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

Motivation & Objective

• Provide a comprehensive taxonomy for the space of AI explainability methods that accounts for diverse consumer personas.
• Develop an extensible open-source toolkit (AIX360) that implements multiple explainability methods across data, model, and prediction stages.
• Enhance existing explainability methods to improve practical usability for data scientists and non-technical stakeholders.
• Offer educational materials, tutorials, and demonstrations to broaden accessibility of explainability concepts across domains.

Proposed method

• Introduce a taxonomy to organize explainability methods by what is explained, how, and at what level (local/global).
• Describe an extensible software architecture that maps taxonomy leaves to specific explainability algorithms within AIX360.
• Implement eight explainability algorithms (e.g., BRCG, GLRM, ProtoDash, ProfWeight, TED, CEM, CEM-MAF, DIP-VAE) and two metrics (Faithfulness, Monotonicity).
• Provide an API design that mirrors scikit-learn, enabling extensions by algorithm developers and reuse across data, models, and predictions explainers.
• Present enhancements to make methods more consumable (e.g., BRCG-light, TED data synthesis) and construct an interactive web demo for non-experts.

Experimental results

Research questions

• RQ1 How can a simple taxonomy help users choose appropriate explainability methods for different applications and stakeholders?
• RQ2 What is a scalable software architecture that integrates diverse explainability techniques across the AI modeling pipeline?
• RQ3 How can existing explainability methods be enhanced for practical deployment and accessibility to non-technical audiences?
• RQ4 Can educational materials and demonstrations broaden adoption of explainability in real-world domains such as lending or healthcare?

Key findings

• Proposed taxonomy serves as a decision-tree to guide method selection based on data/model/prediction needs and local/global, static/interactive considerations.
• AIX360 provides eight explainability methods and two evaluation metrics, and maps them to the taxonomy within an extensible Python architecture.
• The paper demonstrates practical enhancements (e.g., BRCG-light, TED data synthesis) to improve adoption and usability in real-world contexts.
• The toolkit includes educational materials, tutorials, and an interactive web demo to introduce explainability to non-experts across domains.