[Paper Review] Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Arun Das, Paul Rad|arXiv (Cornell University)|Jun 16, 2020

Explainable Artificial Intelligence (XAI)122 references492 citations

TL;DR

This paper surveys the explainable AI landscape for deep learning, offering taxonomies, mathematical summaries of seminal work, evaluation insights, and a timeline of key developments from 2007 to 2020, including an empirical evaluation of eight XAI algorithms on image data.

ABSTRACT

Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In addition to providing a holistic view of the current XAI landscape in deep learning, this paper provides mathematical summaries of seminal work. We start by proposing a taxonomy and categorizing the XAI techniques based on their scope of explanations, methodology behind the algorithms, and explanation level or usage which helps build trustworthy, interpretable, and self-explanatory deep learning models. We then describe the main principles used in XAI research and present the historical timeline for landmark studies in XAI from 2007 to 2020. After explaining each category of algorithms and approaches in detail, we then evaluate the explanation maps generated by eight XAI algorithms on image data, discuss the limitations of this approach, and provide potential future directions to improve XAI evaluation.

Motivation & Objective

• Propose a taxonomy of XAI by scope, methodology, and usage to clarify the explainability landscape.
• Summarize and classify core mathematical models and algorithms in XAI.
• Evaluate explanation maps from eight XAI algorithms on image data to assess limitations and guide future work.
• Discuss evaluation strategies, limitations, and future directions to improve trust, transparency, and fairness in DNN explanations.
• Provide open-source software references and a timeline of landmark XAI studies from 2007 to 2020.

Proposed method

• Taxonomize XAI techniques into local/global scope, and backpropagation-based, perturbation-based, or intrinsic methods.
• Classify methods by usage as model-intrinsic or post-hoc and model-agnostic applicability.
• Provide mathematical overviews of seminal XAI algorithms and standard notations.
• Present and analyze evaluation approaches for XAI explanations and discuss limitations.
• Compare eight XAI algorithms using explanation maps on image data to illustrate practical behavior.
• Offer a timeline of landmark XAI papers and a survey of open-source software implementations.

Experimental results

Research questions

• RQ1 What are the principal taxonomies for explainable AI in deep learning in terms of scope, methodology, and usage?
• RQ2 How can seminal XAI algorithms be mathematically summarized and classified within the proposed taxonomy?
• RQ3 What are the strengths and limitations of current evaluation methods for XAI explanations across modalities?
• RQ4 How do eight representative XAI algorithms perform when generating explanation maps for image data, and what future directions do they suggest?
• RQ5 What open-source tools support XAI methods and how do they align with the surveyed taxonomy?

Key findings

• The authors propose three well-defined taxonomies to organize XAI approaches by scope, methodology, and usage.
• They provide mathematical summaries of seminal XAI methods and a historical timeline from 2007 to 2020.
• They evaluate eight XAI algorithms on image data to illustrate explanation generation and discuss evaluation limitations.
• The survey discusses desirable properties for explanations to improve trust, transparency, and fairness in AI systems.
• The paper highlights the availability of popular open-source software implementations and emphasizes model-agnostic post-hoc explanations as a practical focus.
• A timeline of landmark XAI studies is provided to contextualize rapid developments in the field.