[Paper Review] Interpretable Deep Learning: Interpretations, Interpretability, Trustworthiness, and Beyond.

Xuhong Li, Haoyi Xiong+6 more|arXiv (Cornell University)|Mar 19, 2021

Explainable Artificial Intelligence (XAI)137 references4 citations

TL;DR

This paper provides a comprehensive survey of interpretable deep learning by clarifying core concepts like interpretations and interpretability, proposing a new taxonomy for interpretation algorithms, and evaluating their performance and trustworthiness. It further explores connections with adversarial robustness and data augmentation, and offers open-source tools for implementation.

ABSTRACT

Deep neural networks have been well-known for their superb performance in handling various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal the ways that deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. First of all, to address the research efforts in interpretations, we elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. Then, to understand the results of interpretation, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the existing work in evaluating models' interpretability using trustworthy interpretation algorithms. Finally, we review and discuss the connections between deep models' interpretations and other factors, such as adversarial robustness and data augmentations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.

Motivation & Objective

• To clarify the often-confused distinction between 'interpretations' and 'interpretability' in deep learning.
• To provide a systematic taxonomy of recent interpretation algorithms based on their design principles and objectives.
• To survey and evaluate performance metrics used for assessing interpretation algorithms.
• To examine how trustworthy interpretation methods can enhance model interpretability and reliability.
• To investigate the interplay between model interpretations, adversarial robustness, and data augmentation techniques.

Proposed method

• Proposes a new taxonomy to categorize interpretation algorithms based on their underlying mechanisms and objectives.
• Reviews and classifies various interpretation techniques, such as saliency maps, attention visualization, and concept activation, from multiple perspectives.
• Evaluates interpretation algorithms using standardized performance metrics, including faithfulness, stability, and fidelity.
• Analyzes the role of trustworthy interpretation in improving model transparency and user confidence.
• Explores the impact of adversarial examples and data augmentation on interpretation quality and model behavior.
• Recommends and documents open-source libraries for implementing and evaluating interpretation methods.

Experimental results

Research questions

• RQ1 How do the concepts of 'interpretations' and 'interpretability' differ, and why is this distinction important in deep learning?
• RQ2 What are the key design principles and categories of modern interpretation algorithms, and how can they be systematically classified?
• RQ3 Which performance metrics are most effective for evaluating the quality and reliability of interpretation methods?
• RQ4 How can trustworthy interpretation algorithms enhance the interpretability and trustworthiness of deep models?
• RQ5 What is the relationship between model interpretations and factors such as adversarial robustness and data augmentation?

Key findings

• The distinction between 'interpretations' (model-specific explanations) and 'interpretability' (the model's inherent explainability) is critical for rigorous evaluation.
• A new taxonomy for interpretation algorithms enables clearer understanding and comparison of diverse methods across different design goals.
• Faithfulness, stability, and fidelity are key metrics for evaluating the quality of interpretation outputs.
• Trustworthy interpretation methods significantly improve user confidence and model reliability in real-world applications.
• Adversarial robustness and data augmentation can influence the consistency and reliability of interpretation results.
• Several open-source libraries are available to support the implementation and evaluation of interpretation techniques.