[论文解读] Understanding Student and Academic Staff Perceptions of AI Use in Assessment and Feedback

Jasper Roe, Mike Perkins+1 more|arXiv (Cornell University)|Jun 22, 2024

Online Learning and Analytics被引用 5

一句话总结

本研究调查越南和新加坡的学生与学术人员，以了解对 GenAI 的熟悉度、对评估与反馈中 AI 的看法，以及对文本检测工具和政策影响的观点。

ABSTRACT

The rise of Artificial Intelligence (AI) and Generative Artificial Intelligence (GenAI) in higher education necessitates assessment reform. This study addresses a critical gap by exploring student and academic staff experiences with AI and GenAI tools, focusing on their familiarity and comfort with current and potential future applications in learning and assessment. An online survey collected data from 35 academic staff and 282 students across two universities in Vietnam and one in Singapore, examining GenAI familiarity, perceptions of its use in assessment marking and feedback, knowledge checking and participation, and experiences of GenAI text detection. Descriptive statistics and reflexive thematic analysis revealed a generally low familiarity with GenAI among both groups. GenAI feedback was viewed negatively; however, it was viewed more positively when combined with instructor feedback. Academic staff were more accepting of GenAI text detection tools and grade adjustments based on detection results compared to students. Qualitative analysis identified three themes: unclear understanding of text detection tools, variability in experiences with GenAI detectors, and mixed feelings about GenAI's future impact on educational assessment. These findings have major implications regarding the development of policies and practices for GenAI-enabled assessment and feedback in higher education.

研究动机与目标

• 评估学生与学术人员对生成式 AI（GenAI）的熟悉程度。
• 探讨 GenAI 在评估评分与反馈中的使用看法。
• 检查与 GenAI 相关的知识核对与参与度。
• 调查对 GenAI 文本检测及对评分的政策影响的经验与态度。

提出的方法

• 通过线上调查，面向越南和新加坡的35名学术人员和282名学生发放。
• 用描述性统计来汇总熟悉度、看法与经验。
• 采用反身性主题分析从开放式回答中识别定性主题。

实验结果

研究问题

• RQ1 学生与学术人员对 GenAI 的熟悉程度如何？
• RQ2 学生与学术人员如何看待 GenAI 在评估评分与反馈中的使用？
• RQ3 对 GenAI 文本检测工具及相关评分影响的体验有哪些？
• RQ4 关于 GenAI 对教育评估未来影响的主题有哪些？

主要发现

• GenAI 熟悉度总体上在学生和学术人员中都较低。
• GenAI 反馈在独立使用时被视为负面，但与导师反馈结合时更为积极。
• 相比学生，工作人员更容易接受 GenAI 文本检测工具以及基于检测结果的分数调整。
• 定性主题包括对检测工具缺乏清晰理解、检测工具体验的差异，以及对 GenAI 未来影响的观点不一。