[Paper Review] Explainable Artificial Intelligence (XAI): An Engineering Perspective

Fatima Hussain, Rasheed Hussain+1 more|arXiv (Cornell University)|Jan 10, 2021

Explainable Artificial Intelligence (XAI)Computer Science41 references40 citations

TL;DR

The paper argues for an engineering-centric view of XAI, defines explainability, interpretability, and transparency, outlines stakeholder needs, and presents an autonomous car use-case to illustrate where and how explainability should be applied.

ABSTRACT

The remarkable advancements in Deep Learning (DL) algorithms have fueled enthusiasm for using Artificial Intelligence (AI) technologies in almost every domain; however, the opaqueness of these algorithms put a question mark on their applications in safety-critical systems. In this regard, the `explainability' dimension is not only essential to both explain the inner workings of black-box algorithms, but it also adds accountability and transparency dimensions that are of prime importance for regulators, consumers, and service providers. eXplainable Artificial Intelligence (XAI) is the set of techniques and methods to convert the so-called black-box AI algorithms to white-box algorithms, where the results achieved by these algorithms and the variables, parameters, and steps taken by the algorithm to reach the obtained results, are transparent and explainable. To complement the existing literature on XAI, in this paper, we take an `engineering' approach to illustrate the concepts of XAI. We discuss the stakeholders in XAI and describe the mathematical contours of XAI from engineering perspective. Then we take the autonomous car as a use-case and discuss the applications of XAI for its different components such as object detection, perception, control, action decision, and so on. This work is an exploratory study to identify new avenues of research in the field of XAI.

Motivation & Objective

• Define XAI concepts (explainability, interpretability, transparency) and justify their importance in safety-critical AI.
• Identify stakeholders and explainability requirements for different user groups.
• Provide an engineering/mathematical perspective on XAI, and illustrate with an autonomous car use-case across perception, object detection, and control.

Proposed method

• Classify XAI systems into transparent (interpretable) models and post-hoc explainability methods.
• Describe levels of algorithmic transparency, decomposability, and simulability of models.
• Discuss stakeholder-specific explainability requirements and role-based explainability needs.
• Present mathematical framing for XAI in autonomous vehicles, including object detection and movement prediction pathways.
• Review existing XAI techniques and how they apply to engineering viewpoints.

Experimental results

Research questions

• RQ1 Who are the stakeholders of XAI and what are their explainability requirements?
• RQ2 How can XAI be framed from an engineering and mathematical perspective for safety-critical systems like autonomous cars?
• RQ3 What components of autonomous vehicles benefit most from XAI and which techniques are suitable?
• RQ4 What gaps exist between current XAI practice and engineering needs in critical domains?

Key findings

• XAI comprises explainability, interpretability, and transparency, with transparency encompassing these as a dominant aspect.
• Transparent models (e.g., linear, decision trees, Bayesian networks) offer varying degrees of algorithmic transparency, decomposability, and simulability.
• Post-hoc explainability methods and model induction can be used to derive explanations from black-box models, with attention to local vs. global fidelity.
• An autonomous car use-case demonstrates how XAI considerations span perception, object detection, localization, movement prediction, and route/decision making.
• A mathematical perspective frames XAI in autonomous cars via system of interacting agents and differential equations to model behavior and explanations for specific components.