[Paper Review] "Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans

Vivian Lai, Han Liu+1 more|arXiv (Cornell University)|Jan 14, 2020

Explainable Artificial Intelligence (XAI)62 references4 citations

TL;DR

This paper proposes model-driven tutorials—using explanations from both simple and deep learning models—to help humans understand counterintuitive patterns in machine learning, particularly in deceptive review detection. It finds that tutorials based on simple models significantly improve human performance, even without real-time assistance, highlighting the value of model-driven pedagogy over static explanations.

ABSTRACT

To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans. While existing approaches focus on explaining machine predictions with real-time assistance, we explore model-driven tutorials to help humans understand these patterns in a training phase. We consider both tutorials with guidelines from scientific papers, analogous to current practices of science communication, and automatically selected examples from training data with explanations. We use deceptive review detection as a testbed and conduct large-scale, randomized human-subject experiments to examine the effectiveness of such tutorials. We find that tutorials indeed improve human performance, with and without real-time assistance. In particular, although deep learning provides superior predictive performance than simple models, tutorials and explanations from simple models are more useful to humans. Our work suggests future directions for human-centered tutorials and explanations towards a synergy between humans and AI.

Motivation & Objective

• To investigate whether model-driven tutorials—distinct from real-time explanations—can improve human performance in interpreting machine learning models.
• To compare the effectiveness of tutorials derived from simple models versus deep learning models in helping humans understand counterintuitive patterns.
• To explore how explanations that include reasoning (e.g., 'why' features matter) enhance human hypothesis generation and decision-making.
• To examine the role of trust and model caveats in tutorial design, especially in high-stakes, human-in-the-loop AI applications.
• To identify design principles for interactive, model-driven tutorials that go beyond static feature attribution.

Proposed method

• Conducted large-scale, randomized, pre-registered human subject experiments with 1,000+ participants on Amazon Mechanical Turk and in-person university students.
• Designed two tutorial types: (1) example-driven tutorials using real training data with model explanations, and (2) guideline-driven tutorials extracted from scientific literature on deception detection.
• Used both simple models (e.g., logistic regression) and deep learning models (e.g., BERT) to generate explanations, focusing on feature importance and textual rationales.
• Presented participants with labeled reviews, model predictions, and explanations—either as highlighted words or as reasoning-based justifications.
• Evaluated performance using accuracy in classifying deceptive vs. genuine reviews, with and without real-time assistance.
• Collected qualitative feedback to assess trust, comprehension, and hypothesis generation in response to explanations.

Experimental results

Research questions

• RQ1 Does using model-driven tutorials improve human performance in detecting deceptive reviews compared to no tutorial?
• RQ2 Are tutorials based on simple models more effective than those based on deep learning models in enhancing human understanding and performance?
• RQ3 Does including reasoning (e.g., 'why' certain words are indicative) in tutorials improve human hypothesis generation more than feature attribution alone?
• RQ4 How does the inclusion of model caveats and strengths affect human trust and performance in tutorial settings?
• RQ5 To what extent do interactive or dynamic explanations (e.g., counterfactual manipulation) improve learning compared to static explanations?

Key findings

• Tutorials significantly improved human performance in deceptive review detection, even without real-time assistance, demonstrating the value of a dedicated training phase.
• Despite deep learning models achieving higher predictive accuracy, tutorials based on simple models led to greater improvements in human performance than those based on deep learning models.
• Explanations that included reasoning (e.g., 'imagination leads to vague details') were more effective than feature attribution alone, suggesting that explaining 'why' is crucial for hypothesis generation.
• Tutorials that highlighted model strengths and limitations improved trust and reduced overreliance on model outputs, supporting the need for balanced model communication.
• The effectiveness of tutorials was consistent across demographic groups, though results may not generalize to non-technical populations.
• Interactive tutorials allowing users to test counterfactuals (e.g., removing words) showed potential for deeper understanding, though not fully evaluated in this study.