[Paper Review] Beyond the Hype: A Cautionary Tale of ChatGPT in the Programming Classroom

Grant Oosterwyk, Pitso Tsibolane+2 more|arXiv (Cornell University)|Jun 16, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This study investigates the practical implications of using ChatGPT 3.5 in undergraduate information systems and computer science programming courses, applying Complex Adaptive Systems (CAS) theory to analyze code generation from past student exercises. It reveals common errors and assessment challenges, offering educators strategies to design more robust programming tasks that foster critical coding skills despite AI assistance.

ABSTRACT

Due to the proliferation of Large Language Models research and the use of various Artificial Intelligence (AI) tools, the field of information systems (IS) and computer science (CS) has evolved. The use of tools such as ChatGPT to complete various student programming exercises (e.g., in Python) and assignments has gained prominence amongst various academic institutions. However, recent literature has suggested that the use of ChatGPT in academia is problematic and the impact on teaching and learning should be further scrutinized. More specifically, little is known about how ChatGPT can be practically used with code (programming) writing to complete programming exercises amongst IS and CS undergraduate university students. Furthermore, the paper provides insights for academics who teach programming to create more challenging exercises and how to engage responsibly in the use of ChatGPT to promote classroom integrity. In this paper, we used Complex Adaptive Systems (CAS) theory as a theoretical guide to understand the various dynamics through classroom code demonstrations. Using ChatGPT 3.5, we analyzed the various practical programming examples from past IS exercises and compared those with memos created by tutors and lecturers in a university setting. This paper highlights common ways of assessment, programming errors created by ChatGPT and the potential consideration for IS academics to ensure the development of critical programming skills among students.

Motivation & Objective

• To investigate how ChatGPT 3.5 performs in generating code for real university-level programming exercises in information systems and computer science.
• To identify common programming errors and flawed logic introduced by ChatGPT in student assignments.
• To explore how AI-assisted code generation affects assessment integrity and learning outcomes in programming courses.
• To provide actionable insights for educators to design more resilient programming exercises that promote critical thinking and reduce over-reliance on AI.
• To apply Complex Adaptive Systems (CAS) theory to understand the dynamic interactions between students, AI tools, and teaching practices in programming classrooms.

Proposed method

• Applied Complex Adaptive Systems (CAS) theory as a theoretical framework to analyze emergent behaviors in classroom code generation processes.
• Collected and analyzed past IS programming exercises and corresponding tutor- and lecturer-created memos for consistency and correctness.
• Used ChatGPT 3.5 to generate code solutions for the same exercises used in the original course materials.
• Compared AI-generated code outputs with human-authored solutions to identify discrepancies, logical flaws, and implementation errors.
• Conducted classroom demonstrations to observe real-time interactions between students, AI tools, and teaching staff.
• Evaluated the reliability and pedagogical implications of AI-generated code through qualitative analysis of error patterns and assessment challenges.

Experimental results

Research questions

• RQ1 How does ChatGPT 3.5 perform in generating correct and logically sound code for standard undergraduate programming exercises in IS and CS courses?
• RQ2 What types of programming errors are most commonly introduced by ChatGPT when solving academic coding tasks?
• RQ3 How do AI-generated solutions compare to those created by experienced tutors and lecturers in terms of correctness and pedagogical value?
• RQ4 In what ways does the use of ChatGPT compromise assessment integrity in programming courses?
• RQ5 How can educators redesign programming exercises to minimize over-reliance on AI while preserving learning outcomes?

Key findings

• ChatGPT 3.5 frequently generated code that was syntactically correct but logically flawed, leading to incorrect outputs for common programming tasks.
• Common error types included incorrect loop conditions, improper handling of edge cases, and flawed data structure usage, even when prompts were clear.
• AI-generated solutions often lacked clarity, modularity, and best practices, making them difficult to debug or extend.
• Assessment based on AI-generated code risks rewarding syntactic correctness over conceptual understanding and problem-solving ability.
• Tutors and lecturers’ memos often emphasized algorithmic reasoning and edge-case handling, which were frequently missing in ChatGPT’s outputs.
• The study demonstrates that AI-generated code can undermine learning if not critically evaluated, highlighting the need for pedagogically robust exercise design.