[Paper Review] Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning

Leilani H. Gilpin, David Bau+4 more|arXiv (Cornell University)|May 31, 2018

Explainable Artificial Intelligence (XAI)Computer Science53 references120 citations

TL;DR

This paper proposes a standardized framework for evaluating interpretability in machine learning by defining explainability and classifying existing XAI methods. It identifies shortcomings in current approaches—especially for deep neural networks—and outlines future research directions to improve transparency, fairness, and systematic assessment of explanations.

ABSTRACT

There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we provide our definition of explainability and show how it can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufficient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artificial intelligence.

Motivation & Objective

• To establish a clear, consistent definition of explainability to unify the XAI research field.
• To classify and analyze existing explanatory methods in machine learning, particularly for deep neural networks.
• To identify critical limitations in current XAI approaches that hinder systematic evaluation.
• To propose future research directions that address gaps in interpretability and evaluation standards.

Proposed method

• Proposes a formal definition of explainability as the ability of a system to provide understandable, contextually relevant explanations.
• Classifies existing XAI methods based on their underlying mechanisms and evaluation criteria.
• Analyzes current approaches to explanation generation, especially in deep learning, highlighting inconsistencies and lack of standardization.
• Identifies key challenges such as lack of evaluation benchmarks, inconsistent metrics, and insufficient user-centered design.
• Proposes a framework for systematic assessment of explanations based on transparency, fidelity, and user comprehension.
• Outlines future research directions focused on standardization, evaluation protocols, and integration of human factors in explanation design.

Experimental results

Research questions

• RQ1 How can explainability be formally defined to enable consistent evaluation across XAI methods?
• RQ2 Why are current explanation methods for deep neural networks insufficient for reliable and systematic assessment?
• RQ3 What are the key gaps in the evaluation and standardization of machine learning explanations?
• RQ4 How can explanations be made more transparent, faithful, and useful to end users?
• RQ5 What future research directions are needed to improve the reliability and impact of XAI systems?

Key findings

• The paper identifies a lack of standardized definitions and evaluation criteria as a major barrier to progress in XAI.
• Current XAI methods, especially for deep neural networks, often lack consistency, reproducibility, and user-centered validation.
• There is no unified benchmark or metric to assess the quality of explanations across different models and domains.
• The study reveals that explanations are frequently evaluated based on internal model properties rather than user understanding or decision impact.
• The authors conclude that future work must prioritize systematic evaluation frameworks and human-in-the-loop validation.
• Standardization of terminology, evaluation protocols, and integration of user feedback are essential for advancing trustworthy and interpretable AI.