[Paper Review] Innovating Computer Programming Pedagogy: The AI-Lab Framework for Generative AI Adoption

Ethan Dickey, Andres Bejarano+1 more|arXiv (Cornell University)|Aug 23, 2023

Software Engineering Research4 citations

TL;DR

This paper introduces the AI-Lab framework, a pedagogical model that guides computer science students in responsibly integrating generative AI (GenAI) into foundational programming courses. By emphasizing prompt engineering, error detection, and critical evaluation of GenAI outputs, the framework preserves core skill development—such as problem-solving and algorithmic thinking—while reducing over-reliance, as evidenced by shifts in student perceptions of GenAI reliability post-intervention.

ABSTRACT

Over the last year, the ascent of Generative AI (GenAI) has raised concerns about its impact on core skill development, such as problem-solving and algorithmic thinking, in Computer Science students. Preliminary anonymous surveys show that at least 48.5% of our students use GenAI for homework. With the proliferation of these tools, the academic community must contemplate the appropriate role of these tools in education. Neglecting this might culminate in a phenomenon we term the "Junior-Year Wall," where students struggle in advanced courses due to prior over-dependence on GenAI. Instead of discouraging GenAI use, which may unintentionally foster covert usage, our research seeks to answer: "How can educators guide students' interactions with GenAI to preserve core skill development during their foundational academic years?" We introduce "AI-Lab," a pedagogical framework for guiding students in effectively leveraging GenAI within core collegiate programming courses. This framework accentuates GenAI's benefits and potential as a pedagogical instrument. By identifying and rectifying GenAI's errors, students enrich their learning process. Moreover, AI-Lab presents opportunities to use GenAI for tailored support such as topic introductions, detailed examples, corner case identification, rephrased explanations, and debugging assistance. Importantly, the framework highlights the risks of GenAI over-dependence, aiming to intrinsically motivate students towards balanced usage. This approach is premised on the idea that mere warnings of GenAI's potential failures may be misconstrued as instructional shortcomings rather than genuine tool limitations. Additionally, AI-Lab offers strategies for formulating prompts to elicit high-quality GenAI responses. For educators, AI-Lab provides mechanisms to explore students' perceptions of GenAI's role in their learning experience.

Motivation & Objective

• To address growing concerns about generative AI undermining core programming skills like problem-solving and algorithmic thinking in foundational computer science courses.
• To counter the 'Junior-Year Wall' phenomenon, where students struggle in advanced courses due to over-dependence on GenAI in early semesters.
• To guide educators in transforming GenAI from a potential academic integrity risk into a constructive learning tool through structured pedagogical integration.
• To shift student perception of GenAI from a shortcut to a critical thinking aid by emphasizing error detection, prompt refinement, and contextual understanding.
• To provide educators with tools to assess and shape student perceptions of GenAI's role in learning, based on empirical survey data.

Proposed method

• The AI-Lab framework integrates GenAI as a scaffolded learning partner, encouraging students to use it for topic introductions, code examples, corner case identification, rephrased explanations, and debugging assistance.
• Students are taught systematic prompt engineering techniques to elicit higher-quality, more accurate GenAI responses, reducing hallucinations and errors.
• The framework emphasizes critical evaluation of GenAI outputs, training students to detect and correct errors—turning GenAI use into an active learning process.
• Educators use pre- and post-intervention surveys to assess shifts in student perceptions of GenAI reliability and usage patterns.
• The framework embeds GenAI use within structured course workflows, ensuring it supports, rather than replaces, cognitive skill development.
• The approach avoids blanket prohibitions, instead promoting intrinsic motivation through transparency about GenAI’s limitations and strengths.

Experimental results

Research questions

• RQ1 How can educators guide students to use generative AI in ways that preserve core programming skills like problem-solving and algorithmic thinking?
• RQ2 What impact does structured GenAI integration have on student perceptions of reliability and over-reliance?
• RQ3 In what ways can GenAI be leveraged as a pedagogical tool for personalized support without undermining foundational learning?
• RQ4 How can prompt engineering and error detection be taught as central components of effective GenAI use in programming education?
• RQ5 What role does student perception play in shaping responsible GenAI adoption, and how can educators influence this constructively?

Key findings

• After implementing the AI-Lab framework, 6.9% of students rated GenAI as 'Very reliable' (5 on a 5-point scale), up from 2.8% in the pre-survey, indicating a measurable shift toward greater trust in GenAI's utility.
• The proportion of students rating GenAI as 'Very unreliable' (1) dropped from 0% to 10.3%, suggesting increased confidence in the tool’s outputs after guided use.
• A significant 37.1% of students expressed uncertainty about peer GenAI usage, highlighting the prevalence of ambiguous or unreported tool use in academic settings.
• The framework successfully shifted student perception from viewing GenAI as a potential shortcut to seeing it as a tool requiring critical evaluation and refinement.
• Students who engaged with the AI-Lab framework demonstrated improved ability to identify and correct GenAI-generated errors, reinforcing cognitive engagement over passive acceptance.
• The study confirms that structured GenAI integration can reduce the risk of the 'Junior-Year Wall' by fostering metacognitive skills and responsible tool use from early semesters.