[Paper Review] Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation

Brett A. Becker, Paul Denny+4 more|arXiv (Cornell University)|Dec 2, 2022

Teaching and Learning ProgrammingComputer Science36 citations

TL;DR

This position paper discusses opportunities and challenges of AI-driven code generation tools (Codex, AlphaCode, CodeWhisperer) for introductory programming education and urges rapid adaptation of teaching practices. It surveys implications for learning, pedagogy, ethics, and assessment as AI-generated code becomes more available.

ABSTRACT

The introductory programming sequence has been the focus of much research in computing education. The recent advent of several viable and freely-available AI-driven code generation tools present several immediate opportunities and challenges in this domain. In this position paper we argue that the community needs to act quickly in deciding what possible opportunities can and should be leveraged and how, while also working on how to overcome or otherwise mitigate the possible challenges. Assuming that the effectiveness and proliferation of these tools will continue to progress rapidly, without quick, deliberate, and concerted efforts, educators will lose advantage in helping shape what opportunities come to be, and what challenges will endure. With this paper we aim to seed this discussion within the computing education community.

Motivation & Objective

• Assess how AI code generation tools affect introductory programming learning and assessment.
• Identify opportunities to enhance learning resources, exemplars, and diverse solutions.
• Highlight ethical, integrity, and sustainability concerns for computing education.
• Propose pedagogical shifts to leverage AI code generation while mitigating risks.

Proposed method

• Survey of existing AI code generation tools (Codex/OpenAI Codex, AlphaCode, CodeWhisperer) and their capabilities.
• Synthesis of prior computational education literature on CS1/CS2 pedagogy and assessment.
• Discussion of opportunities and challenges grounded in cited studies and tool documentation.
• propositions for new instructional approaches and resource generation strategies.

Experimental results

Research questions

• RQ1 What opportunities do AI-driven code generation tools offer for learning, teaching, and assessment in introductory programming?
• RQ2 What challenges (academic integrity, bias, security, sustainability) arise from these tools in CS1/CS2 settings?
• RQ3 How might pedagogy shift to leverage code generation while maintaining core learning outcomes and ethical practices?

Key findings

• AI-generated code can provide exemplar solutions and expose solution variety for learning and assessment.
• Code generation can aid in producing learning resources like exercises, explanations, and worked examples at scale.
• New pedagogies may shift focus toward higher-level algorithmic understanding and code review over writing every line of code.
• There are significant ethical, licensing, and security concerns that require proactive guidance and oversight in education.