[Paper Review] Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks

Roberto Confalonieri, Tillman Weyde+2 more|arXiv (Cornell University)|Jun 19, 2019

Explainable Artificial Intelligence (XAI)Computer Science44 references19 citations

TL;DR

This paper proposes Trepan Reloaded, a knowledge-driven extension of the Trepan algorithm that enhances the understandability of global post-hoc explanations from artificial neural networks by integrating domain-specific ontologies during decision tree extraction. The method improves user comprehension in finance and medical domains without significantly compromising model fidelity or accuracy.

ABSTRACT

Explainability in Artificial Intelligence has been revived as a topic of active research by the need of conveying safety and trust to users in the `how' and `why' of automated decision-making. Whilst a plethora of approaches have been developed for post-hoc explainability, only a few focus on how to use domain knowledge, and how this influences the understandability of global explanations from the users' perspective. In this paper, we show how ontologies help the understandability of global post-hoc explanations, presented in the form of symbolic models. In particular, we build on Trepan, an algorithm that explains artificial neural networks by means of decision trees, and we extend it to include ontologies modeling domain knowledge in the process of generating explanations. We present the results of a user study that measures the understandability of decision trees using a syntactic complexity measure, and through time and accuracy of responses as well as reported user confidence and understandability. The user study considers domains where explanations are critical, namely, in finance and medicine. The results show that decision trees generated with our algorithm, taking into account domain knowledge, are more understandable than those generated by standard Trepan without the use of ontologies.

Motivation & Objective

• To address the gap in explainable AI regarding the integration of domain knowledge to improve explanation understandability.
• To investigate whether using ontologies during decision tree generation enhances users’ comprehension of black-box model explanations.
• To evaluate the impact of ontology-informed explanations on user performance, confidence, and perceived understandability in high-stakes domains.
• To maintain high fidelity and accuracy of the surrogate decision trees while improving their interpretability through semantic enrichment.

Proposed method

• Extends the original Trepan algorithm to incorporate domain knowledge by modifying split selection logic to prioritize features linked to more general concepts in a predefined ontology.
• Uses ontologies to enrich decision tree nodes with semantic information, improving the interpretability of symbolic explanations.
• Applies a syntactic complexity measure to evaluate the technical understandability of generated decision trees.
• Conducts a user study comparing decision trees generated with and without ontology integration, measuring response time, accuracy, confidence, and self-reported understandability.
• Focuses on two high-stakes domains—finance and medicine—where explanation clarity is critical for trust and decision-making.
• Employs a controlled experimental design with real users to assess cognitive and perceptual impacts of ontology-enhanced explanations.

Experimental results

Research questions

• RQ1 Does integrating domain knowledge via ontologies improve the understandability of global post-hoc explanations generated from neural networks?
• RQ2 How does ontology-informed feature selection affect the syntactic complexity and cognitive load of decision trees?
• RQ3 To what extent does the use of ontologies enhance user accuracy, response time, and self-reported confidence in interpreting model decisions?
• RQ4 Can ontology-based explanations maintain high fidelity and accuracy compared to standard Trepan-generated trees?

Key findings

• Decision trees generated by Trepan Reloaded, which incorporate domain ontologies, were significantly more understandable than those from standard Trepan, as measured by user-reported understandability and confidence.
• Users answered questions faster and more accurately when explanations were enriched with ontology-based semantic information, indicating reduced cognitive load.
• The syntactic complexity of the decision trees was lower when ontologies were used, suggesting improved structural clarity and readability.
• The enhanced understandability was achieved with minimal compromise on the accuracy and fidelity of the surrogate decision trees compared to the original neural network.
• The results demonstrate that ontology integration improves explanation quality without sacrificing model performance, particularly in high-stakes domains like medicine and finance.
• The study confirms that domain knowledge, when structured via ontologies, plays a crucial role in shaping more interpretable and user-friendly explanations.