[Paper Review] GPTutor: a ChatGPT-powered programming tool for code explanation

Eason Chen, Ray Huang+3 more|arXiv (Cornell University)|May 3, 2023

Scientific Computing and Data Management16 citations

TL;DR

GPTutor is a Visual Studio Code extension that uses the ChatGPT API to provide code explanations by analyzing code with source reference, delivering concise explanations within a pop-up, and publicly available on GitHub and VS Code Marketplace.

ABSTRACT

Learning new programming skills requires tailored guidance. With the emergence of advanced Natural Language Generation models like the ChatGPT API, there is now a possibility of creating a convenient and personalized tutoring system with AI for computer science education. This paper presents GPTutor, a ChatGPT-powered programming tool, which is a Visual Studio Code extension using the ChatGPT API to provide programming code explanations. By integrating Visual Studio Code API, GPTutor can comprehensively analyze the provided code by referencing the relevant source codes. As a result, GPTutor can use designed prompts to explain the selected code with a pop-up message. GPTutor is now published at the Visual Studio Code Extension Marketplace, and its source code is openly accessible on GitHub. Preliminary evaluation indicates that GPTutor delivers the most concise and accurate explanations compared to vanilla ChatGPT and GitHub Copilot. Moreover, the feedback from students and teachers indicated that GPTutor is user-friendly and can explain given codes satisfactorily. Finally, we discuss possible future research directions for GPTutor. This includes enhancing its performance and personalization via further prompt programming, as well as evaluating the effectiveness of GPTutor with real users.

Motivation & Objective

• Motivate the need for tailored, AI-assisted programming tutoring in computer science education.
• Develop a tool that integrates ChatGPT within Visual Studio Code to explain code with reference to relevant source code.
• Enable concise, accurate, and user-friendly code explanations accessible via a pop-up in the IDE.
• Provide open-source access to the tool and demonstrate its potential via preliminary user and teacher feedback.

Proposed method

• Integrate the Visual Studio Code API with the ChatGPT API to create GPTutor as a programming explanation tool.
• Use designed prompts and code-reference mechanisms to generate explanations that reference relevant source code.
• Present explanations to users through a pop-up message within the IDE.
• Publish the tool on the Visual Studio Code Extension Marketplace and release the source code on GitHub.

Experimental results

Research questions

• RQ1 Does GPTutor provide more concise and accurate explanations than vanilla ChatGPT and GitHub Copilot?
• RQ2 Is GPTutor user-friendly for students and teachers?
• RQ3 Can GPTutor be effectively extended and personalized through prompt programming?
• RQ4 What is the potential impact of GPTutor on learning outcomes in programming education?

Key findings

• Preliminary evaluation indicates GPTutor yields more concise explanations compared to baselines (vanilla ChatGPT and GitHub Copilot).
• Feedback from students and teachers suggests GPTutor is user-friendly and satisfactorily explains given code.
• GPTutor’s architecture supports integration with IDEs and access to source code for contextualized explanations.
• The paper discusses future directions for improving performance, personalization, and real-user evaluation.