[論文レビュー] Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges

Basit Qureshi|arXiv (Cornell University)|Apr 16, 2023

Artificial Intelligence in Healthcare and Education被引用数 57

ひとこと要約

本論文は、2年生の Data Structures and Algorithms コースにおける ChatGPT の有無による学習成果を実験的に比較し、ChatGPT はスコアに寄与する一方でコードの一貫性の欠如や不正確さを招くことを示しています。

ABSTRACT

The application of Artificial intelligence for teaching and learning in the academic sphere is a trending subject of interest in the computing education. ChatGPT, as an AI-based tool, provides various advantages, such as heightened student involvement, cooperation, accessibility and availability. This paper addresses the prospects and obstacles associated with utilizing ChatGPT as a tool for learning and assessment in undergraduate Computer Science curriculum in particular to teaching and learning fundamental programming courses. Students having completed the course work for a Data Structures and Algorithms (a sophomore level course) participated in this study. Two groups of students were given programming challenges to solve within a short period of time. The control group (group A) had access to text books and notes of programming courses, however no Internet access was provided. Group B students were given access to ChatGPT and were encouraged to use it to help solve the programming challenges. The challenge was conducted in a computer lab environment using PC2 environment. Each team of students address the problem by writing executable code that satisfies certain number of test cases. Student teams were scored based on their performance in terms of number of successful passed testcases. Results show that students using ChatGPT had an advantage in terms of earned scores, however there were inconsistencies and inaccuracies in the submitted code consequently affecting the overall performance. After a thorough analysis, the paper's findings indicate that incorporating AI in higher education brings about various opportunities and challenges.

研究の動機と目的

• AIツール（ChatGPT）を学部レベルのCS教育に統合する機会と課題を探る。
• 学生がChatGPTを用いてプログラミング課題に取り組んだ場合の学習および評価結果を評価する。
• AI支援を用いて作成された学生提出コードの正確性と信頼性を評価する。
• AI支援学習におけるカリキュラム設計と教育戦略に関する洞察を提供する。

提案手法

• Two groups of Data Structures and Algorithms students (sophomore level) participated in programming challenges.
• Group A (control) used textbooks and notes with no Internet access.
• Group B had access to ChatGPT and were encouraged to use it to solve challenges.
• Challenges were performed in a PC2 computer lab environment with executable code judged by test cases.
• Teams were scored based on the number of test cases their code passed.
• Comparison focused on performance scores and observed code quality.

実験結果

リサーチクエスチョン

• RQ1 Does access to ChatGPT improve student performance on programming challenges compared to traditional resources?
• RQ2 What are the benefits and drawbacks of using ChatGPT for learning and assessment in undergraduate CS?
• RQ3 How does AI-assisted coding affect code correctness and reliability in student submissions?

主な発見

• Students using ChatGPT earned higher scores than those without AI assistance.
• There were inconsistencies and inaccuracies in ChatGPT-generated code that affected overall performance.
• AI-assisted learning introduces opportunities (involvement, accessibility) and challenges (code reliability) in higher education.