[论文解读] Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience

Bhavya Ghai, Q. Vera Liao+3 more|arXiv (Cornell University)|Jan 24, 2020

Explainable Artificial Intelligence (XAI)参考文献 103被引用 18

一句话总结

本文提出可解释主动学习（XAL），一种新颖的人机交互范式，将局部模型解释整合至主动学习界面中，以提升机器教学效果。通过在请求标注前向标注者展示模型的预测结果及其推理依据，XAL增强了信任校准并支持更丰富的反馈，但存在锚定偏差风险，并增加了认知参与度较低用户的认知负担。

ABSTRACT

The wide adoption of Machine Learning technologies has created a rapidly growing demand for people who can train ML models. Some advocated the term "machine teacher" to refer to the role of people who inject domain knowledge into ML models. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query the machine teacher for labels. However, in current AL settings, the human-AI interface remains minimal and opaque. We begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the human teacher should be able to understand the reasoning underlying the mistakes. When the model matures, the machine teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without the explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and need for cognition. By reflecting on the results, we suggest future directions and design implications for XAL.

研究动机与目标

• 探究模型解释如何影响人类‘机器教师’在主动学习中的体验与表现。
• 评估解释是否能提升主动学习界面中的信任校准与反馈质量。
• 识别个体因素（如任务知识、AI经验、认知需求）在用户对解释响应中的调节作用。
• 探讨将解释作为人机界面核心组件在机器教学中设计的启示。
• 比较XAL与传统主动学习及协同学习在学习成效与用户体验方面的差异。

提出的方法

• XAL将局部解释方法（如LIME或类似特征重要性解释）整合至主动学习循环中，用于在查询人类标注前解释模型预测结果。
• 参与者与一个模型交互，该模型选择不确定样本，展示预测结果与解释，随后请求标注与反馈。
• 本研究比较三种条件：传统主动学习（无解释）、协同学习（仅展示预测）、XAL（展示预测+解释）。
• 标注者提供标签与对模型预测的反馈，反馈类型与内容通过定性与定量方法进行分析。
• 模型性能在迭代过程中被追踪，信任校准通过自我报告的自信度与测试性能的一致性进行测量。
• 统计分析评估了不同条件下学习效率、反馈质量与认知负荷的差异。

实验结果

研究问题

• RQ1 模型解释如何影响人类标注者在主动学习过程中的信任校准？
• RQ2 当提供模型解释时，人类标注者自然会提供哪些类型的反馈？
• RQ3 解释的存在如何影响机器教师的认知工作量与标注准确性？
• RQ4 哪些个体差异（如任务知识、AI经验、认知需求）会调节基于解释的教学效果？
• RQ5 XAL在学习效率与反馈丰富度方面，与传统主动学习及协同学习相比有何差异？

主要发现

• XAL显著提升了标注者校准信任的能力，表现为自我报告的自信度与实际模型性能之间的一致性更高。
• 与其它条件相比，XAL条件下标注者提供了更多样化且更具洞察力的反馈，包括对模型推理过程的修正与基于特征层面的批判性意见。
• 解释的使用导致了明显的锚定效应，即当解释看似合理时，标注者更可能接受错误的模型预测，尤其在任务知识较低的用户中更为显著。
• 认知需求较低的参与者在XAL中报告了更高的认知工作量，表明基于解释的界面可能并非对所有用户都具有普适可及性。
• 随着模型逐渐成熟，XAL中的不确定性采样变得日益吃力，原因在于模型对错误预测的置信度不断提高，凸显了对具备解释感知能力的采样策略的迫切需求。
• 本研究发现，有效的XAL不仅依赖于准确的解释，还需考虑标注者专业水平与反馈倾向的解释设计策略。