[Paper Review] (Un)reasonable Allure of Ante-hoc Interpretability for High-stakes Domains: Transparency Is Necessary but Insufficient for Comprehensibility

Kacper Sokol, Julia E. Vogt|arXiv (Cornell University)|Jun 4, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper critiques the overreliance on ante-hoc interpretability in high-stakes domains like healthcare, arguing that transparency alone is insufficient for comprehensibility. It proposes a human-centred framework integrating modularity, provenance, lineage, and reasoning types to ensure explanations are truly understandable across diverse audiences.

ABSTRACT

Ante-hoc interpretability has become the holy grail of explainable artificial intelligence for high-stakes domains such as healthcare; however, this notion is elusive, lacks a widely-accepted definition and depends on the operational context. It can refer to predictive models whose structure adheres to domain-specific constraints, or ones that are inherently transparent. The latter conceptualisation assumes observers who judge this quality, whereas the former presupposes them to have technical and domain expertise (thus alienating other groups of explainees). Additionally, the distinction between ante-hoc interpretability and the less desirable post-hoc explainability, which refers to methods that construct a separate explanatory model, is vague given that transparent predictive models may still require (post-)processing to yield suitable explanatory insights. Ante-hoc interpretability is thus an overloaded concept that comprises a range of implicit properties, which we unpack in this paper to better understand what is needed for its safe adoption across high-stakes domains. To this end, we outline modelling and explaining desiderata that allow us to navigate its distinct realisations in view of the envisaged application and audience.

Motivation & Objective

• To address the ambiguity and lack of consensus around ante-hoc interpretability in high-stakes domains such as healthcare.
• To identify why inherently transparent models may still fail to be comprehensible to non-expert explainees.
• To propose a structured framework of desiderata—modularity, provenance, lineage, and reasoning—that enhances comprehensibility beyond transparency.
• To clarify the distinction between ante-hoc and post-hoc explainability by focusing on the source and processing of explanatory insights.
• To guide the design of XAI systems that are not only transparent but also meaningfully understandable to diverse audiences.

Proposed method

• Unpacks the concept of ante-hoc interpretability by distinguishing it from post-hoc methods through functional, structural, and audience-based criteria.
• Introduces a framework based on four key desiderata: modularity of explanations, provenance of explanatory information (endogenous vs. exogenous), lineage of insights (translucency), and reasoning type (human, algorithmic, or hybrid).
• Analyzes how explanatory insights derive from model structure (endogenous) or surrogate models (exogenous), emphasizing truthfulness and traceability.
• Evaluates reasoning processes that interpret model outputs, such as rule interpretation, counterfactual generation, and coefficient analysis, to assess their cognitive load and interpretive accuracy.
• Proposes a spectrum-based approach to explainability that maps techniques along the provenance–lineage axis, distinguishing high-translucency ante-hoc methods from low-translucency post-hoc ones.
• Uses case studies from healthcare and decision trees to illustrate how model transparency does not guarantee user comprehension, especially when reasoning is misattributed or oversimplified.

Experimental results

Research questions

• RQ1 Why is ante-hoc interpretability often perceived as superior despite its lack of a stable, widely-accepted definition?
• RQ2 To what extent does model transparency ensure comprehensibility for non-expert explainees in high-stakes domains?
• RQ3 How do provenance and lineage of explanatory insights affect the trustworthiness and reliability of explanations?
• RQ4 What role does reasoning—human, algorithmic, or hybrid—play in transforming model outputs into meaningful explanations?
• RQ5 How can XAI systems be designed to ensure that explanations are not only transparent but also comprehensible across diverse audiences?

Key findings

• Ante-hoc interpretability is an overloaded and ambiguous concept, often conflating transparency with comprehensibility, leading to misaligned expectations in high-stakes domains.
• Even inherently transparent models can fail to be understood by non-expert users due to cognitive mismatches, such as misinterpreting root splits in decision trees as indicators of feature importance.
• Endogenous explanations derived directly from a transparent model have higher provenance reliability and translucency than exogenous ones, which rely on surrogate models and risk inaccuracy.
• The reasoning process behind an explanation—whether human, algorithmic, or hybrid—significantly influences its comprehensibility and must be aligned with the explainee’s expertise.
• Provenance and lineage of explanations are critical for trust, with endogenous insights offering higher transparency and lower risk of distortion compared to post-hoc surrogate models.
• A human-centred approach that incorporates modularity, audience-specific reasoning, and traceable information lineage is essential to bridge the gap between transparency and actual comprehensibility.