[Paper Review] Directions for Explainable Knowledge-Enabled Systems

Shruthi Chari, Daniel M. Gruen+2 more|arXiv (Cornell University)|Mar 17, 2020

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes a taxonomy of nine explanation types for explainable knowledge-enabled AI systems, emphasizing user-centric, context-aware, and provenance-enabled explanations. It integrates techniques like neuro-symbolic reasoning, semantic representations, and distributed ledger technology (DLT) to enhance trust, comprehensibility, and causality in AI explanations, advancing hybrid systems that align with user needs and situational contexts.

ABSTRACT

Interest in the field of Explainable Artificial Intelligence has been growing for decades and has accelerated recently. As Artificial Intelligence models have become more complex, and often more opaque, with the incorporation of complex machine learning techniques, explainability has become more critical. Recently, researchers have been investigating and tackling explainability with a user-centric focus, looking for explanations to consider trustworthiness, comprehensibility, explicit provenance, and context-awareness. In this chapter, we leverage our survey of explanation literature in Artificial Intelligence and closely related fields and use these past efforts to generate a set of explanation types that we feel reflect the expanded needs of explanation for today's artificial intelligence applications. We define each type and provide an example question that would motivate the need for this style of explanation. We believe this set of explanation types will help future system designers in their generation and prioritization of requirements and further help generate explanations that are better aligned to users' and situational needs.

Motivation & Objective

• Address the growing need for explainable AI in complex, high-stakes domains by focusing on user comprehension, context, and trust.
• Identify limitations in current AI explanation approaches that rely solely on mechanistic or model-specific reasoning.
• Develop a comprehensive, user-oriented explanation framework that integrates diverse explanation types for real-world AI applications.
• Advance hybrid AI systems by combining techniques such as causal reasoning, semantic modeling, and DLT for provenance and trust.
• Provide a structured methodology for system designers to select and prioritize explanation types based on user needs and situational context.

Proposed method

• Synthesize a taxonomy of nine explanation types from literature in AI, cognitive science, and social sciences, focusing on user context and comprehensibility.
• Integrate neuro-symbolic AI methods to combine symbolic reasoning with machine learning interpretability for causal and transparent explanations.
• Leverage semantic representations to model knowledge and explanation structures, enabling context-aware and granular explanations.
• Apply distributed ledger technology (DLT) to ensure provenance, traceability, and trust in AI decisions, especially in decentralized or adversarial environments.
• Use case studies from health informatics (HEALS project) and autonomous systems (UAVs, swarm robotics) to validate explanation types in real-world settings.
• Propose hybrid explanation architectures that combine multiple explanation types (e.g., causal, contrastive, counterfactual) to meet diverse user needs and situational demands.

Figure 1 : A partial conceptual framework mapping explanation types capable of being generated by explainable AI methods to the reasoning employed by users [Image taken from Lim et al. [ 7 ] and Wang et al. [ 8 ] ]. For the full framework see Wang et al. [ 8 ] .

Experimental results

Research questions

• RQ1 What types of explanations are most effective in supporting user understanding, trust, and decision-making in complex AI systems?
• RQ2 How can explanation systems be designed to adapt to user context, preferences, and situational needs beyond mechanistic model tracing?
• RQ3 In what ways can neuro-symbolic AI and semantic modeling improve the interpretability and coherence of explanations?
• RQ4 How can distributed ledger technology (DLT) enhance the provenance and trustworthiness of AI decisions, especially in decentralized or adversarial environments?
• RQ5 What hybrid combinations of explanation types are most effective for real-world AI applications requiring both transparency and usability?

Key findings

• The paper identifies and defines nine distinct explanation types—such as causal, contrastive, counterfactual, and procedural—that are tailored to different user needs and contexts.
• Current AI systems often fail to provide explanations that are comprehensible or actionable for end-users, even when they offer model-level transparency.
• Hybrid explanation systems that combine multiple explanation types (e.g., causal + counterfactual) are more effective in supporting joint human-AI reasoning and decision-making.
• Distributed ledger technology (DLT) enables verifiable, immutable provenance for AI decisions, enhancing trust in decentralized or high-risk environments like UAVs and IoT systems.
• Semantic representations of explanations improve context-awareness and enable dynamic adaptation of explanation form and granularity based on user profile and situation.
• Neuro-symbolic methods bridge the gap between black-box model behavior and symbolic reasoning, enabling explanations that are both interpretable and grounded in domain knowledge.

Figure 2 : A representation of Pearl’s cause-effect model [ 32 , 33 ] where $Q=P(Y|do(X))$ , where $X$ has an effect on $Y$ and both depend on Z. Hence, he formulated the overall problem as a Bayesian equation in that $E_{z}=\sum_{z}P(Y|X,Z)P(Z)$ . Pearl provides an intuitive example [ 32 ] of gende