[Paper Review] A Meta Survey of Quality Evaluation Criteria in Explanation Methods

Helena Löfström, Karl Hammar+1 more|arXiv (Cornell University)|Mar 25, 2022

Explainable Artificial Intelligence (XAI)26 references4 citations

TL;DR

This paper proposes using 'appropriate trust' as a measurable outcome metric to enable comparative evaluation of explanation methods in XAI. By analyzing 15 literature surveys, it identifies four core criteria—performance, appropriate trust, explanation satisfaction, and fidelity—across three quality aspects (model, explanation, user), offering a unified model to standardize evaluation and overcome subjectivity in comparative research.

ABSTRACT

Explanation methods and their evaluation have become a significant issue in explainable artificial intelligence (XAI) due to the recent surge of opaque AI models in decision support systems (DSS). Since the most accurate AI models are opaque with low transparency and comprehensibility, explanations are essential for bias detection and control of uncertainty. There are a plethora of criteria to choose from when evaluating explanation method quality. However, since existing criteria focus on evaluating single explanation methods, it is not obvious how to compare the quality of different methods. This lack of consensus creates a critical shortage of rigour in the field, although little is written about comparative evaluations of explanation methods. In this paper, we have conducted a semi-systematic meta-survey over fifteen literature surveys covering the evaluation of explainability to identify existing criteria usable for comparative evaluations of explanation methods. The main contribution in the paper is the suggestion to use appropriate trust as a criterion to measure the outcome of the subjective evaluation criteria and consequently make comparative evaluations possible. We also present a model of explanation quality aspects. In the model, criteria with similar definitions are grouped and related to three identified aspects of quality; model, explanation, and user. We also notice four commonly accepted criteria (groups) in the literature, covering all aspects of explanation quality: Performance, appropriate trust, explanation satisfaction, and fidelity. We suggest the model be used as a chart for comparative evaluations to create more generalisable research in explanation quality.

Motivation & Objective

• Address the lack of consensus and standardization in evaluating explanation methods in XAI.
• Identify commonly accepted evaluation criteria across existing surveys to enable comparative assessments.
• Overcome the challenge of subjective user evaluations by proposing 'appropriate trust' as an objective outcome metric.
• Develop a structured model of explanation quality integrating criteria across model, explanation, and user aspects.
• Provide a framework for generalizable, comparable evaluations of explanation methods in XAI research.

Proposed method

• Conducted a semi-systematic meta-survey across 15 literature surveys on explanation method evaluation.
• Mapped and grouped evaluation criteria from the surveys into coherent categories based on shared definitions and purposes.
• Identified three core aspects of explanation quality: model, explanation, and user, and related criteria to these aspects.
• Proposed 'appropriate trust' as a measurable outcome of subjective user criteria, enabling objective comparison.
• Developed a high-level model of explanation quality integrating 11 identified criteria groups across the three aspects.
• Validated the model's utility by demonstrating that four criteria—performance, appropriate trust, explanation satisfaction, and fidelity—appear in over half of the surveyed works and span all three quality aspects.

Experimental results

Research questions

• RQ1 Which evaluation criteria are most consistently used across literature surveys on explanation methods?
• RQ2 How can subjective user evaluation criteria be transformed into objective, comparable metrics for explanation method evaluation?
• RQ3 What are the core aspects of explanation quality, and how do evaluation criteria relate to them?
• RQ4 Can a unified model of explanation quality be constructed from existing criteria to support comparative evaluations?
• RQ5 To what extent do performance, fidelity, explanation satisfaction, and appropriate trust serve as foundational criteria for evaluating explanation methods?

Key findings

• Four criteria—performance, appropriate trust, explanation satisfaction, and fidelity—are consistently mentioned in more than half of the 15 surveyed literature reviews, indicating broad consensus on their importance.
• The meta-survey identified 11 distinct evaluation criterion groups, which were organized into three quality aspects: model, explanation, and user.
• The criterion 'appropriate trust' was identified as a key outcome metric that can objectively measure the success of subjective user evaluation criteria, enabling cross-method comparison.
• The study found that existing evaluation practices are heavily focused on human-in-the-loop assessments, creating challenges for reproducibility and comparability across studies.
• The proposed model of explanation quality provides a structured framework to align diverse evaluation criteria and support more generalizable research in XAI.
• The research highlights the need for standardized definitions and benchmarks, as many criteria (e.g., reliability, confidence, certainty) are used interchangeably or ambiguously across studies.