[Paper Review] Integrating ChatGPT in a Computer Science Course: Students Perceptions and Suggestions

Kehinde Aruleba, Ismaila Temitayo Sanusi+2 more|arXiv (Cornell University)|Dec 22, 2023

Artificial Intelligence in Healthcare and Education4 citations

TL;DR

This study investigates undergraduate computer science students' perceptions of integrating ChatGPT into a course through code completion and analysis tasks. Findings reveal that students value ChatGPT for instant feedback and personalized learning, but express concerns about reduced critical thinking and response inconsistency, highlighting the need for balanced, pedagogically guided integration in higher education.

ABSTRACT

The integration of artificial intelligence tools such as ChatGPT in the education system has gained attention in recent years. This experience report explores students' perceptions and suggestions for integrating ChatGPT in a computer science course. Following a ChatGPT activity which includes code completion and analysis, seven students participated in in-depth interviews. Findings from the transcribed interviews suggest that ChatGPT has the potential to enhance learning experience including programming. They highlighted the tool's ability to respond immediately to queries and supporting personalised learning. However, they raise concerns that heavy reliance on ChatGPT may adversely affect students' critical thinking and problem-solving skills. These findings show the importance of carefully balancing using ChatGPT in computer science courses. The findings of this research have significant implications for educators, curriculum designers and policymakers as they explore integrating AI tools into educational contexts.

Motivation & Objective

• To understand how computer science students perceive the integration of ChatGPT in a university-level course.
• To examine the impact of ChatGPT on students’ learning experiences, particularly in programming tasks.
• To identify students’ concerns regarding overreliance on AI tools and data privacy in educational settings.
• To gather student suggestions for improving the use of ChatGPT in teaching and learning programming.

Proposed method

• Conducted a ChatGPT-based activity involving code completion and analysis in a computer science course.
• Selected seven students for in-depth, semi-structured interviews following the activity.
• Transcribed and analyzed interview data using qualitative thematic analysis to identify recurring themes.
• Focused on students’ experiences, perceived benefits, challenges, and suggestions for AI integration.
• Explored the role of ChatGPT in enhancing self-directed learning and code refactoring.
• Assessed concerns related to response inconsistency, critical thinking erosion, and data privacy.

Figure 1: P6 - ChatGPT generated code with comments

Experimental results

Research questions

• RQ1 How do computer science students perceive the use of ChatGPT in a programming course?
• RQ2 What benefits do students identify in using ChatGPT for learning programming?
• RQ3 What challenges or concerns do students express regarding reliance on ChatGPT?
• RQ4 How do students suggest improving the integration of ChatGPT in educational settings?
• RQ5 What are the implications of ChatGPT use for critical thinking and academic integrity in CS education?

Key findings

• Students reported that ChatGPT enhanced their learning experience by providing immediate feedback and supporting personalized, self-directed learning.
• Participants viewed ChatGPT as useful for code restructuring, aligning with goals of improving code readability and maintainability.
• Students expressed concern that overreliance on ChatGPT could impair the development of critical thinking and problem-solving skills.
• Inconsistencies in ChatGPT’s responses were noted as a significant challenge, potentially reinforcing misinformation if not critically evaluated.
• Data privacy emerged as a key concern, with students questioning how personal information is collected, stored, and used by the AI tool.
• Students recommended increased customization, interactivity, and transparency in AI tool design to improve educational effectiveness and trust.

Figure 2: P4 - ChatGPT explaining the output of a code