[Paper Review] Evaluating Language Models for Generating and Judging Programming Feedback

Charles Koutcheme, Nicola Dainese+5 more|arXiv (Cornell University)|Jul 5, 2024

Software Engineering Techniques and Practices4 citations

TL;DR

This study evaluates open-source large language models (LLMs) for generating and judging programming feedback on Python assignments, comparing them to proprietary models like GPT-4o. Results show that state-of-the-art open-source models like Llama3-70B perform nearly as well as proprietary models in both feedback generation and quality assessment, demonstrating strong potential for cost-effective, transparent, and accessible educational tools.

ABSTRACT

The emergence of large language models (LLMs) has transformed research and practice across a wide range of domains. Within the computing education research (CER) domain, LLMs have garnered significant attention, particularly in the context of learning programming. Much of the work on LLMs in CER, however, has focused on applying and evaluating proprietary models. In this article, we evaluate the efficiency of open-source LLMs in generating high-quality feedback for programming assignments and judging the quality of programming feedback, contrasting the results with proprietary models. Our evaluations on a dataset of students' submissions to introductory Python programming exercises suggest that state-of-the-art open-source LLMs are nearly on par with proprietary models in both generating and assessing programming feedback. Additionally, we demonstrate the efficiency of smaller LLMs in these tasks and highlight the wide range of LLMs accessible, even for free, to educators and practitioners.

Motivation & Objective

• To assess the quality of feedback generated by open-source LLMs for introductory Python programming assignments.
• To evaluate whether open-source LLMs can effectively judge the quality of programming feedback, comparable to human experts.
• To compare the performance of open-source and proprietary LLMs in both feedback generation and assessment tasks.
• To explore the feasibility of using smaller, freely available LLMs for educational feedback applications.
• To promote transparency, cost efficiency, and data privacy in computing education by advocating for open-source LLMs over proprietary alternatives.

Proposed method

• Generated bug explanations and suggested fixes using five state-of-the-art open-source LLMs and two proprietary models (including GPT-4o).
• Manually evaluated feedback quality using a custom rubric focusing on completeness, comprehensibility, and fix accuracy.
• Used expert-annotated ground truth labels to evaluate model performance in judging feedback quality.
• Employed the EasyLLM library and HuggingFace APIs for model access, with minimal cost for most models.
• Conducted inter-rater reliability checks with two annotators, achieving a moderate Krippendorff’s alpha of 0.49.
• Evaluated model performance across diverse LLM sizes, including Phi-3-mini, Mistral-7B, and Llama3-70B, to assess scalability and efficiency.

Experimental results

Research questions

• RQ1 RQ1: How do open- and closed-source LLMs compare in generating high-quality bug explanations and fixes for student programming submissions?
• RQ2 RQ2: To what extent can open- and closed-source LLMs accurately assess the quality of programming feedback relative to expert human judgment?
• RQ3 RQ3: How do smaller, freely accessible LLMs perform in feedback generation and judging tasks compared to larger models?
• RQ4 RQ4: What are the trade-offs in performance, cost, and data privacy when using open-source versus proprietary LLMs in educational feedback systems?

Key findings

• Llama3-70B, a state-of-the-art open-source LLM, achieved performance on par with GPT-4o in both feedback generation and judging tasks.
• Smaller open-source models like Phi-3-mini demonstrated competitive performance despite limited computational resources, suggesting viability for deployment on consumer devices.
• The inter-rater reliability for human annotation was moderate (Krippendorff’s alpha = 0.49), indicating room for improvement in annotation consistency.
• Open-source LLMs showed strong potential as LLM-judges, with performance comparable to GPT-4o in feedback assessment, enabling iterative feedback improvement.
• The study found that model size does not directly correlate with performance, as smaller models like Phi-3-mini outperformed expectations.
• The authors released code, model outputs, and annotations to support reproducibility and future benchmarking in the LLM-for-education space.