[Paper Review] An Evaluation of the Human-Interpretability of Explanation

Isaac Lage, Emily Chen+5 more|arXiv (Cornell University)|Jan 31, 2019

Explainable Artificial Intelligence (XAI)Computer Science59 references121 citations

TL;DR

The paper empirically studies how different types of explanation complexity in decision sets affect human interpretability across tasks and domains, identifying cognitive chunks as a key driver of usability.

ABSTRACT

Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations interpretable under three specific tasks that users may perform with machine learning systems: simulation of the response, verification of a suggested response, and determining whether the correctness of a suggested response changes under a change to the inputs. Through carefully controlled human-subject experiments, we identify regularizers that can be used to optimize for the interpretability of machine learning systems. Our results show that the type of complexity matters: cognitive chunks (newly defined concepts) affect performance more than variable repetitions, and these trends are consistent across tasks and domains. This suggests that there may exist some common design principles for explanation systems.

Motivation & Objective

• Investigate what makes explanations human-interpretable for common ML tasks.
• Quantify how explanation properties (size, cognitive chunks, repetition) impact usability.
• Compare interpretability across two domains (recipe recommendations and clinical decisions) and three tasks (simulation, verification, counterfactual).
• Identify regularizers that improve interpretability for decision-set explanations.

Proposed method

• Construct controlled, hand-crafted decision-set explanations to mimic machine-learned outputs.
• Manipulate three explanation variation dimensions: size, cognitive chunks, and repeated terms.
• Evaluate across two domains (recipe and clinical) and three tasks (simulation, verification, counterfactual).
• Measure performance via three metrics: accuracy, response time, and subjective satisfaction.
• Recruit 150 subjects per experiment from MTurk and apply inclusion criteria based on practice questions.

Experimental results

Research questions

• RQ1 Which properties of decision-set explanations most affect human usability across tasks and domains?
• RQ2 Do cognitive chunks, line/term length, or repetition influence response time, accuracy, and satisfaction differently?
• RQ3 Are there domain- and task-general design principles for interpretable explanations?

Key findings

• Greater explanation complexity generally increases response time across tasks and domains.
• Cognitive chunks (new concepts) have a larger impact on performance than mere repetition of terms.
• Explicitly defined cognitive chunks tend to increase response time more than implicitly embedded chunks, suggesting scan/processing costs.
• The effects of explanation size (lines and output terms) on response time vary by domain; in recipes, these effects are more pronounced.
• Repeated terms showed less consistent and smaller impact on response time than introducing new cognitive chunks.