[Paper Review] Tell Me a Story! Narrative-Driven XAI with Large Language Models

David Martens, James Hinns+3 more|arXiv (Cornell University)|Sep 29, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces XAIstories, a narrative-driven approach to Explainable AI (XAI) that leverages Large Language Models (LLMs) to generate human-friendly, natural language explanations from SHAP and counterfactual (CF) explanations. It demonstrates that over 90% of lay users find SHAPstories convincing, and CFstories outperform manually written narratives in speed (10x faster) and accuracy (20%+ improvement), offering a compelling solution to the interpretability gap in AI systems.

ABSTRACT

In many AI applications today, the predominance of black-box machine learning models, due to their typically higher accuracy, amplifies the need for Explainable AI (XAI). Existing XAI approaches, such as the widely used SHAP values or counterfactual (CF) explanations, are arguably often too technical for users to understand and act upon. To enhance comprehension of explanations of AI decisions and the overall user experience, we introduce XAIstories, which leverage Large Language Models to provide narratives about how AI predictions are made: SHAPstories do so based on SHAP explanations, while CFstories do so for CF explanations. We study the impact of our approach on users' experience and understanding of AI predictions. Our results are striking: over 90% of the surveyed general audience finds the narratives generated by SHAPstories convincing. Data scientists primarily see the value of SHAPstories in communicating explanations to a general audience, with 83% of data scientists indicating they are likely to use SHAPstories for this purpose. In an image classification setting, CFstories are considered more or equally convincing as the users' own crafted stories by more than 75% of the participants. CFstories additionally bring a tenfold speed gain in creating a narrative. We also find that SHAPstories help users to more accurately summarize and understand AI decisions, in a credit scoring setting we test, correctly answering comprehension questions significantly more often than they do when only SHAP values are provided. The results thereby suggest that XAIstories may significantly help explaining and understanding AI predictions, ultimately supporting better decision-making in various applications.

Motivation & Objective

• To address the gap in human-friendly, coherent explanations for complex AI models, especially in critical domains where interpretability is essential.
• To overcome the limitations of SHAP and counterfactual explanations—such as technical complexity and lack of narrative coherence—by transforming them into natural language stories.
• To evaluate whether LLM-generated narratives (XAIstories) are perceived as more convincing, accurate, and efficient than manually crafted explanations by both lay users and data scientists.
• To assess the potential of XAIstories to enhance communication of AI decisions to non-expert audiences and improve trust and understanding in AI systems.

Proposed method

• XAIstories leverages fine-tuned Large Language Models (LLMs) to convert SHAP values into narrative explanations (SHAPstories) that interpret feature contributions to a prediction score.
• For counterfactual explanations, XAIstories generates narratives (CFstories) that explain the minimal changes to input features that would alter the predicted class, framing them as 'what if' scenarios.
• The method uses prompt engineering and structured input templates derived from SHAP and CF outputs to guide the LLM in generating coherent, factual, and contextually relevant narratives.
• The approach is evaluated through user studies with both lay participants and data scientists, comparing LLM-generated stories against manually written narratives and baseline explanations.
• Evaluation metrics include perceived credibility, narrative quality, speed of creation, and accuracy of explanation content, with statistical analysis to validate findings.
• The framework is applied to tabular data (student performance prediction) and image classification (MobileNet V2 misclassification), demonstrating cross-domain applicability.

Experimental results

Research questions

• RQ1 Can LLM-generated narratives (XAIstories) significantly improve the perceived clarity and credibility of SHAP explanations among non-expert users?
• RQ2 How do CFstories compare to manually written narratives in terms of perceived quality, accuracy, and creation speed?
• RQ3 To what extent do data scientists perceive XAIstories as valuable for communicating AI explanations to non-technical stakeholders?
• RQ4 Can narrative explanations derived from SHAP and counterfactuals enhance user trust and understanding of AI predictions in real-world decision-making contexts?

Key findings

• Over 90% of the general audience found SHAPstories to be convincing, indicating strong user acceptance of LLM-generated narratives for SHAP explanations.
• 92% of data scientists reported that SHAPstories would improve the ease and confidence of nonspecialists in understanding AI predictions.
• 83% of data scientists indicated a high likelihood of using SHAPstories to communicate explanations to non-expert audiences.
• In image classification, CFstories were considered more or equally convincing as user-crafted narratives by over 75% of lay participants.
• CFstories reduced narrative creation time by a factor of ten compared to manual writing, demonstrating a significant efficiency gain.
• CFstories improved narrative accuracy by over 20% compared to manually created explanations, suggesting better fidelity to the underlying counterfactual logic.