[论文解读] Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments

Vuthea Chheang, Shayla Sharmin+8 more|arXiv (Cornell University)|Jun 29, 2023

Artificial Intelligence in Healthcare and Education被引用 12

一句话总结

本研究提出一个具有生成式 AI 化身虚拟助手的 VR 解剖教育环境（化身 vs 屏幕），并在一个包含两种认知复杂度水平的两组Pilot中评估用户表现、可用性与沉浸感，参与者共16人。

ABSTRACT

Virtual reality (VR) and interactive 3D visualization systems have enhanced educational experiences and environments, particularly in complicated subjects such as anatomy education. VR-based systems surpass the potential limitations of traditional training approaches in facilitating interactive engagement among students. However, research on embodied virtual assistants that leverage generative artificial intelligence (AI) and verbal communication in the anatomy education context is underrepresented. In this work, we introduce a VR environment with a generative AI-embodied virtual assistant to support participants in responding to varying cognitive complexity anatomy questions and enable verbal communication. We assessed the technical efficacy and usability of the proposed environment in a pilot user study with 16 participants. We conducted a within-subject design for virtual assistant configuration (avatar- and screen-based), with two levels of cognitive complexity (knowledge- and analysis-based). The results reveal a significant difference in the scores obtained from knowledge- and analysis-based questions in relation to avatar configuration. Moreover, results provide insights into usability, cognitive task load, and the sense of presence in the proposed virtual assistant configurations. Our environment and results of the pilot study offer potential benefits and future research directions beyond medical education, using generative AI and embodied virtual agents as customized virtual conversational assistants.

研究动机与目标

• 通过将生成式 AI 与具象化 VR 相结合以支持口头互动和自适应提问，推动解剖教育的发展。
• 在 VR 中评估基于化身和屏幕的 AI 助手在不同认知复杂度下的技术效能与可用性。
• 在被试内设计中评估用户体验指标（可用性、任务负荷、沉浸感）与表现。
• 识别 AI 驱动的 VR 解剖教育的优势、局限性与未来研究方向。

提出的方法

• 开发了基于 Unity 的沉浸式 VR 解剖环境，配备由 ChatGPT-3.5 驱动的具象虚拟助手。
• 在一个 2x2 被试内设计中比较两种交互配置（带口型同步语音的化身 vs 纯文本屏幕）。
• 为问题实现两个认知层级（知识型和分析型，基于布鲁姆分类学）。
• 收集客观绩效数据（任务完成时间、交互次数、分数）以及主观测量（SUS、NASA TLX、IPQ）。
• 使用双因素方差分析（two-way ANOVA）分析配置、认知层级和交互效应。

Figure 1 : System architecture of our proposed VR environment with the generative AI-based embodied virtual assistant.

实验结果

研究问题

• RQ1 RQ1 基于化身的与基于屏幕的虚拟助手配置如何影响解剖教育中的用户表现？
• RQ2 RQ2 主观测量（可用性、任务负荷、沉浸感）如何与虚拟助手配置相关？
• RQ3 RQ3 使用生成式 AI 进行解剖教育的优势、局限性和潜在研究方向是什么？

主要发现

• 化身配置总体得分更高，并且知识型与分析型问题之间的变化更大（在化身中的认知水平显著差异，F(1,15)=4.62, p=0.046, ηp2=0.07）。
• 与分析型问题相比，化身配置的知识型问题回答更快、交互次数更少，表明认知复杂性效应。
• 两种配置的可用性都很高（化身 SUS 75.16，屏幕 SUS 76.72），两者之间无显著差异。
• 任务负荷在各配置之间无显著差异，且心理需求在两者中均为最高评定维度。
• 沉浸感得分在各配置间相似，IPQ 结果无显著差异。
• 定性反馈显示对化身与屏幕的偏好不一，部分参与者提出改进语音同步和问题措辞策略的请求。

Figure 2 : Overview of the study procedure. The order of the conditions (marked with ∗ ) was counterbalanced.