[论文解读] Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges

Basit Qureshi|arXiv (Cornell University)|Apr 16, 2023

Artificial Intelligence in Healthcare and Education被引用 57

一句话总结

本文在一门大二数据结构与算法课程中对比在有无ChatGPT的学习成果，结果显示ChatGPT有助于提高分数，但在代码中存在不一致性和不准确性。

ABSTRACT

The application of Artificial intelligence for teaching and learning in the academic sphere is a trending subject of interest in the computing education. ChatGPT, as an AI-based tool, provides various advantages, such as heightened student involvement, cooperation, accessibility and availability. This paper addresses the prospects and obstacles associated with utilizing ChatGPT as a tool for learning and assessment in undergraduate Computer Science curriculum in particular to teaching and learning fundamental programming courses. Students having completed the course work for a Data Structures and Algorithms (a sophomore level course) participated in this study. Two groups of students were given programming challenges to solve within a short period of time. The control group (group A) had access to text books and notes of programming courses, however no Internet access was provided. Group B students were given access to ChatGPT and were encouraged to use it to help solve the programming challenges. The challenge was conducted in a computer lab environment using PC2 environment. Each team of students address the problem by writing executable code that satisfies certain number of test cases. Student teams were scored based on their performance in terms of number of successful passed testcases. Results show that students using ChatGPT had an advantage in terms of earned scores, however there were inconsistencies and inaccuracies in the submitted code consequently affecting the overall performance. After a thorough analysis, the paper's findings indicate that incorporating AI in higher education brings about various opportunities and challenges.

研究动机与目标

• 探索将AI工具（ChatGPT）融入本科CS教育的机会与挑战。
• 评估当学生使用ChatGPT解决编程挑战时的学习与评估结果。
• 评估在AI协助下提交的代码的准确性和可靠性。
• 为AI辅助学习中的课程设计与教学策略提供见解。

提出的方法

• 两组数据结构与算法课程的学生（大二水平）参与编程挑战。
• Group A (control) 使用教材和笔记且不接入互联网。
• Group B 有权限使用 ChatGPT，并被鼓励使用它来解决挑战。
• 挑战在 PC2 计算机实验室环境中进行，代码通过测试用例来判定可执行性。
• 团队的评分基于他们的代码通过的测试用例数量。
• 比较聚焦于表现分数和观测到的代码质量。

实验结果

研究问题

• RQ1 与传统资源相比，是否有使用ChatGPT能提高学生在编程挑战中的表现？
• RQ2 在本科CS学习与评估中使用ChatGPT有哪些利弊？
• RQ3 AI辅助编码如何影响学生提交代码的正确性和可靠性？

主要发现

• 使用ChatGPT的学生获得的分数高于未使用AI辅助的学生。
• 存在ChatGPT生成的代码的不一致性和不准确性，影响了整体表现。
• AI辅助学习在高等教育中带来机会（参与度、可获得性）和挑战（代码可靠性）。