[Paper Review] AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling

Cristina Conati, Kaśka Porayska‐Pomsta+1 more|arXiv (Cornell University)|Jun 30, 2018

Intelligent Tutoring Systems and Adaptive Learning18 references48 citations

TL;DR

The paper argues that interpretable/open learner models (OLMs) are essential for trustworthy and effective ML in AI in Education, and presents a framework and empirical insights from ITS/OLM research to guide interpretable AI design in high-stakes domains.

ABSTRACT

Interpretability of the underlying AI representations is a key raison d'être for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in ITS (also known as AI in Education) produced important work, which informs about how AI can be used in Education to best effects and, through the OLM research, what are the necessary considerations to make it interpretable and explainable for the benefit of learning. We argue that this work can provide a valuable starting point for a framework of interpretable AI, and as such is of relevance to the application of both knowledge-based and machine learning systems in other high-stakes contexts, beyond education.

Motivation & Objective

• Motivate the need for interpretable AI in education due to high-stakes pedagogical interventions.
• Survey Open Learner Modelling (OLM) as a pathway to interpretable student models.
• Show how scrutable, cooperative, and editable OLMs influence pedagogy, trust, and learning outcomes.
• Provide a conceptual framework and design dimensions to balance user control, explanations, and model transparency.

Proposed method

• Describe Open Learner Modelling (OLM) types: scrutable, cooperative/negotiable, and editable.
• Present concrete ITS/OLM systems as exemplars (e.g., TARDIS scrutable OLM; Long & Aleven 2017 self-assessment with skill bars; Basu et al. 2017 editable OLM).
• Discuss Bayesian Knowledge Tracing visualizations and example-tracing approaches as interpretable representations.
• Summarize empirical findings from ITS literature on user trust, engagement, and learning outcomes in relation to OLM type.
• Propose four design dimensions (from Bull & Kay 2016) to tailor OLM interpretability: purpose, accessible aspects, access/manipulation, and intended users.

Experimental results

Research questions

• RQ1 What is the role of interpretability and explainability in Open Learner Modelling for education?
• RQ2 How do different OLM modalities (scrutable, negotiable, editable) impact learner trust, engagement, and learning outcomes?
• RQ3 What design considerations (dimensions) best support interpretable AI in educational contexts and beyond?
• RQ4 Can insights from OSS/OLM frameworks inform broader interpretable AI practices in high-stakes domains?

Key findings

• OLMs can improve pedagogical outcomes and learner reflection when their assessments are accessible and interpretable.
• Negotiable/co-operative OLMs tend to sustain user trust and engagement by balancing system explanations with user justification.
• Editable OLMs may reduce trust among novice users unless accompanied by targeted system support and explanations.
• Examples from ITS (e.g., TARDIS, example-tracing with Bayesian Knowledge Tracing, skillometer) demonstrate how visualisations and interactions support self-assessment and learning.
• A four-dimensional framework helps calibrate how, why, and to whom model interpretability should be exposed, guiding design choices for interpretable AI in education.