[Paper Review] Semantic XAI for contextualized demand forecasting explanations

Jože M. Rožanec, Dunja Mladenić|arXiv (Cornell University)|Apr 1, 2021

Semantic Web and Ontologies37 references4 citations

TL;DR

This paper proposes a semantic XAI architecture that enhances demand forecasting explanations by integrating domain knowledge, media events, and external datasets via a knowledge graph. It enables actionable, context-aware explanations without exposing model internals, achieving 97% average precision for media event relevance and 56% accuracy in external dataset recommendations.

ABSTRACT

The paper proposes a novel architecture for explainable AI based on semantic technologies and AI. We tailor the architecture for the domain of demand forecasting and validate it on a real-world case study. The provided explanations combine concepts describing features relevant to a particular forecast, related media events, and metadata regarding external datasets of interest. The knowledge graph provides concepts that convey feature information at a higher abstraction level. By using them, explanations do not expose sensitive details regarding the demand forecasting models. The explanations also emphasize actionable dimensions where suitable. We link domain knowledge, forecasted values, and forecast explanations in a Knowledge Graph. The ontology and dataset we developed for this use case are publicly available for further research.

Motivation & Objective

• Address the need for interpretable, context-rich explanations in industrial demand forecasting to support informed decision-making.
• Overcome limitations of black-box AI models by providing local, feature-level explanations that avoid exposing sensitive model details.
• Integrate real-world contextual cues—such as media events and external datasets—into forecast explanations to improve decision support.
• Develop a reusable, publicly available knowledge graph and dataset to advance research in explainable demand forecasting.
• Enable users to identify actionable factors and data enrichment opportunities to improve future forecasting accuracy.

Proposed method

• Design a modular XAI architecture that combines semantic technologies with AI models to generate explanations for demand forecasts.
• Construct a unified ontology to represent domain knowledge, including features, events, and external datasets, enabling semantic proximity detection.
• Implement a Media Event Retrieval System to link forecast-driving features to relevant news articles and extract keywords for contextual relevance.
• Use knowledge graph reasoning to recommend external datasets that could improve forecasting models, based on semantic alignment with forecast features.
• Apply embedding-based similarity measures to rank media events and dataset metadata for inclusion in explanations.
• Manually annotate 528 media events, 401 dataset entries, and 168 forecast explanations to evaluate explanation quality and relevance.

Experimental results

Research questions

• RQ1 How can semantic technologies be leveraged to generate context-aware, actionable explanations for AI-driven demand forecasts?
• RQ2 To what extent do retrieved media events and external datasets enhance the interpretability and usefulness of forecast explanations?
• RQ3 Can a knowledge graph-based approach effectively link forecast features to relevant real-world events and data sources without exposing model internals?
• RQ4 How accurate and relevant are the recommended external datasets and media events in supporting forecast understanding and improvement?
• RQ5 What is the impact of integrating semantic context (e.g., news, events) on the quality and trustworthiness of demand forecasting explanations?

Key findings

• The system achieved 97% average precision@3 for media event relevance, indicating high accuracy in identifying contextually relevant news for forecasts.
• 69% of retrieved media events were judged meaningful in explaining specific forecast features, demonstrating strong contextual alignment.
• 64% of the 401 external dataset metadata entries were found relevant to the forecasting context, supporting data enrichment opportunities.
• The external dataset recommendation system achieved 56% accuracy, with a RDE (Relevance Degree Estimate) of 0.41, indicating moderate but meaningful relevance.
• Media event keywords achieved 78% average precision@3, showing effective semantic summarization of news content for explanation purposes.
• Notable insights emerged from annotations, such as the relevance of ozone pollution data to automotive demand and wireless charging news to EV market trends, validating contextual reasoning.