[Paper Review] Knowledge-based XAI through CBR: There is more to explanations than models can tell

Rosina O. Weber, Manil Shrestha+1 more|arXiv (Cornell University)|Aug 23, 2021

Explainable Artificial Intelligence (XAI)47 references4 citations

TL;DR

This paper proposes Knowledge-based Explainable AI (KBXAI) using Case-Based Reasoning (CBR) to enhance explanations beyond what machine learning models alone can provide. By integrating domain knowledge and subject matter expert insights as additional features, KBXAI improves classification accuracy of explanation categories—demonstrated by a 25% to 32.5% accuracy increase in image data and a 63.43% to 72.64% improvement in text data—showing that model-extrinsic knowledge significantly enhances explainability for diverse stakeholders.

ABSTRACT

The underlying hypothesis of knowledge-based explainable artificial intelligence is the data required for data-centric artificial intelligence agents (e.g., neural networks) are less diverse in contents than the data required to explain the decisions of such agents to humans. The idea is that a classifier can attain high accuracy using data that express a phenomenon from one perspective whereas the audience of explanations can entail multiple stakeholders and span diverse perspectives. We hence propose to use domain knowledge to complement the data used by agents. We formulate knowledge-based explainable artificial intelligence as a supervised data classification problem aligned with the CBR methodology. In this formulation, the inputs are case problems composed of both the inputs and outputs of the data-centric agent and case solutions, the outputs, are explanation categories obtained from domain knowledge and subject matter experts. This formulation does not typically lead to an accurate classification, preventing the selection of the correct explanation category. Knowledge-based explainable artificial intelligence extends the data in this formulation by adding features aligned with domain knowledge that can increase accuracy when selecting explanation categories.

Motivation & Objective

• Address the limitation of current XAI methods that rely solely on model internals, which often fail to capture diverse stakeholder perspectives.
• Overcome the gap between model-based explanations and the rich, context-dependent information needed by human users.
• Develop a framework that integrates domain knowledge and expert insights into explanation selection to improve accuracy and relevance.
• Demonstrate the effectiveness of KBXAI across heterogeneous data types—tabular, image, and text—using a unified CBR-based approach.
• Establish a method for generating explanation categories grounded in domain expertise, rather than relying only on model outputs or data patterns.

Proposed method

• Formulate KBXAI as a supervised classification problem within the CBR framework, where case problems include model inputs and outputs, and case solutions are explanation categories derived from domain knowledge.
• Define explanation categories through expert input and clustering of explanation embeddings, using cosine similarity to identify representative categories.
• Enhance model performance by adding knowledge-based features—such as saliency maps, prototype similarity (e.g., truck/frog), and structural features (e.g., eye/nose triangle) from commonsense or domain knowledge.
• Use nominal and continuous features derived from both model outputs (e.g., saliency) and external knowledge (e.g., prototype embeddings) to extend input representations.
• Apply case extension learning: train a classifier on extended features to predict the most appropriate explanation category for each instance.
• Employ embedding-based similarity measures (e.g., cosine similarity) to map instances to explanation categories and filter low-confidence or redundant categories.

Experimental results

Research questions

• RQ1 How can domain knowledge and expert insights be systematically integrated into XAI to improve explanation quality beyond model-based attribution?
• RQ2 What types of knowledge-based features enhance the accuracy of explanation category selection in diverse data modalities?
• RQ3 How does the inclusion of model-extrinsic knowledge affect the performance of explanation classification in tabular, image, and text data?
• RQ4 Can a CBR-based framework effectively unify explanation generation across heterogeneous data types while maintaining interpretability and accuracy?
• RQ5 What is the relative contribution of model-derived features versus knowledge-derived features in improving explanation selection accuracy?

Key findings

• Adding knowledge-based features such as frog similarity (32.5% accuracy) and dog-specific structural features (e.g., eye/nose triangle) significantly improved explanation classification beyond the baseline of 24% in image data.
• In the image task, the combination of model-derived saliency and knowledge-based features (e.g., prototype similarity) increased accuracy from 24% to 25% and 32.5%, respectively, with the latter being extrinsic to both data and model.
• KBXAI achieved a 14.5% improvement in accuracy (from 63.43% to 72.64%) in the text-based citation recommender task using only two knowledge-based features, despite a binary classification setup.
• Explanation categories were derived from expert input and clustering of explanation embeddings, with the 12 most popular categories selected based on coverage of test instances.
• Features from commonsense knowledge—such as prototype similarity for animals not present in training data—demonstrated strong performance, proving their value in model-extrinsic explanation enhancement.
• Despite the success of knowledge-based features, the method remains limited by reliance on nominal features and the scalability of knowledge acquisition, highlighting the need for automated knowledge learning in future work.