[Paper Review] TED: Teaching AI to Explain its Decisions

Michael Hind, Dennis Wei+6 more|arXiv (Cornell University)|Nov 12, 2018

Explainable Artificial Intelligence (XAI)Computer Science32 references17 citations

TL;DR

This paper introduces TED (Teaching Explanations for Decisions), a framework that trains machine learning models to jointly predict decisions and human-understandable explanations by learning from labeled examples of decisions paired with explanations. Unlike model-introspection approaches, TED aligns explanations with the cognitive complexity and domain knowledge of end users, achieving high prediction accuracy without sacrificing performance in two real-world examples.

ABSTRACT

Artificial intelligence systems are being increasingly deployed due to their potential to increase the efficiency, scale, consistency, fairness, and accuracy of decisions. However, as many of these systems are opaque in their operation, there is a growing demand for such systems to provide explanations for their decisions. Conventional approaches to this problem attempt to expose or discover the inner workings of a machine learning model with the hope that the resulting explanations will be meaningful to the consumer. In contrast, this paper suggests a new approach to this problem. It introduces a simple, practical framework, called Teaching Explanations for Decisions (TED), that provides meaningful explanations that match the mental model of the consumer. We illustrate the generality and effectiveness of this approach with two different examples, resulting in highly accurate explanations with no loss of prediction accuracy for these two examples.

Motivation & Objective

• Address the growing societal and regulatory demand for AI systems to provide 'meaningful information' about their decisions, particularly in high-stakes domains.
• Overcome the limitations of model-introspection approaches that expose internal model mechanics, which are often incomprehensible to non-expert users.
• Develop a practical, generalizable framework that generates explanations tailored to the cognitive complexity and domain understanding of the end user.
• Ensure that explanation generation does not compromise the prediction accuracy of the underlying machine learning model.
• Enable scalable and flexible explanation generation by decoupling explanation format from model architecture and allowing diverse explanation types.

Proposed method

• Train a joint classifier on a dataset where each sample includes input features (X), a decision label (Y), and a human-provided explanation (E), forming a joint prediction task (Y, E).
• Use a Cartesian product of class labels (Y) and explanation options (E) to create a unified multi-class classification problem for training.
• Employ any standard supervised learning algorithm (e.g., random forest, neural network) to learn the joint mapping from input X to decision Y and explanation E.
• Allow explanations to be in any format—textual, symbolic, or structured—provided they are enumerable and semantically distinct.
• Support explanation personalization by enabling multiple explanation variants for the same decision, tailored to different user expertise levels.
• Leverage techniques such as word embeddings or domain-specific languages (DSLs) to manage explanation diversity and improve generalization across similar explanations.

Experimental results

Research questions

• RQ1 How can AI systems generate explanations that are not only accurate but also meaningful and understandable to non-expert users?
• RQ2 Can a machine learning model be trained to produce explanations that align with the cognitive complexity and domain knowledge of the intended audience?
• RQ3 What is the impact of jointly training for decision and explanation on model prediction accuracy compared to traditional model-explanation separation?
• RQ4 How can explanations be efficiently generated and scaled across diverse user profiles and application domains?
• RQ5 What strategies can be used to construct or enrich explanation datasets when full human-annotated explanations are not available?

Key findings

• The TED framework successfully generates explanations that are highly aligned with the mental models of end users, increasing trust and interpretability without reducing prediction accuracy.
• In two real-world examples, the joint prediction model achieved high accuracy in both decision and explanation generation, with no degradation in performance compared to standard models.
• The approach demonstrates generality and simplicity, working effectively across different domains and explanation formats, including free-text and structured explanations.
• The framework supports personalization by enabling multiple explanation variants for the same decision, suitable for users of varying expertise levels.
• The Cartesian product approach to joint learning scales effectively in practice, even with a large number of explanation options, and remains computationally feasible.
• Future extensions such as using external knowledge sources, few-shot learning, or active learning can help reduce the annotation burden for explanation data collection.