[论文解读] A Preliminary Exploration of the Disruption of a Generative AI Systems: Faculty/Staff and Student Perceptions of ChatGPT and its Capability of Completing Undergraduate Engineering Coursework

Lance White, Trini Balart+3 more|arXiv (Cornell University)|Mar 3, 2024

Artificial Intelligence in Healthcare and Education被引用 5

一句话总结

本论文研究 ChatGPT 在完成本科工程课程作业方面的能力，并探讨教职员工与学生的看法，使用 DANCE 模型在混乱中指导教师适应。

ABSTRACT

The authors of this study aim to assess the capabilities of the OpenAI ChatGPT tool to understand just how effective such a system might be for students to utilize in their studies as well as deepen understanding of faculty/staff and student perceptions about ChatGPT in general. The purpose of what is learned from the study is to continue the design of a model to facilitate the development of faculty for becoming adept at embracing change, the DANCE model (Designing Adaptations for the Next Changes in Education). This model is used in this study to help faculty with examining the impact that a disruptive new tool, such as ChatGPT, can pose for the learning environment. The authors analyzed the performance of ChatGPT used to complete course assignments at a variety of levels by novice engineering students working as research assistants. Those completed works have been assessed by the faculty who created those assignments to understand how these completed assignments might compare with the performance of a typical student. A set of surveys conducted by the authors of this work are discussed where students, faculty, and staff respondents in March of 2023 addressed their perceptions of ChatGPT (A follow-up survey is being administered now, February 2024). These survey instruments were analyzed, and the data visualized in this work to bring attention to relevant findings by the researchers. This work reports the findings of the researchers with the purpose of sharing the current state of this work at Texas A&M University with the intention to provide insights to scholars both at our own institution and around the world. This work is not intended to be a finished work but reports these findings with full transparency that this work is currently continuing as the researchers gather new data and develop and validate various measurement instruments.

研究动机与目标

• 评估 OpenAI 的 ChatGPT 在多个层级使用学生研究助理完成工程课程作业的能力。
• 了解教职员工和学生对 ChatGPT 的看法及其对学习环境的影响。
• 利用并进一步发展 DANCE 模型（Designing Adaptations for the Next Changes in Education，面向教育未来变革的适应设计）以指导教师为应对颠覆性工具的准备。
• 在数据收集持续进行的同时，分享来自德州农工大学的当前发现，以支持全球学者。

提出的方法

• 评估由新手工程学生研究助理在不同课程层级生成的 ChatGPT 作业。
• 请创建作业的教师评估 ChatGPT 输出，以与典型学生表现进行比较。
• 对学生、教职员工进行调查（2023年3月）以捕捉对 ChatGPT 的看法；2024年2月进行了后续调查。
• 应用 DANCE 模型来框架教育中的颠覆与适应需求的分析。

实验结果

研究问题

• RQ1 如由教职员工所观察，ChatGPT 在完成本科工程课程作业方面的能力如何？
• RQ2 学生、教职员工如何看待 ChatGPT 及其对学习环境的影响？
• RQ3 如何利用 DANCE 模型为面临生成式 AI 工具颠覆的教育者设计适应方案？
• RQ4 将 ChatGPT 产出与同一作业的典型学生表现进行比较时，出现了哪些洞见？
• RQ5 随着机构持续数据收集和工具验证，预计会有哪些变化？

主要发现

• 初步发现着重指出关于 ChatGPT 及其在工程教育中的颠覆潜力的显著看法。
• 由作业作者将 ChatGPT 输出与典型学生表现进行对比，提供了对能力与局限的早期洞见。
• 调查数据可视化强调关于采用、担忧和教学影响的有意义的趋势与主题。
• 该研究明确指出这不是一项完成的工作，数据收集和工具验证仍在进行。
• 这一工作为德州农工大学更广泛的努力贡献力量，向全球学术界分享早期见解。