[Paper Review] Can ChatGPT pass the Vietnamese National High School Graduation Examination?

Xuan-Quy Dao, Ngoc-Bich Le+2 more|arXiv (Cornell University)|Jun 15, 2023

Artificial Intelligence in Healthcare and Education32 citations

TL;DR

The paper tests ChatGPT on the Vietnamese National High School Graduation Examination (VNHSGE), showing ChatGPT can pass with an average 6–7 across subjects and varying performance by subject.

ABSTRACT

This research article highlights the potential of AI-powered chatbots in education and presents the results of using ChatGPT, a large language model, to complete the Vietnamese National High School Graduation Examination (VNHSGE). The study dataset included 30 essays in the literature test case and 1,700 multiple-choice questions designed for other subjects. The results showed that ChatGPT was able to pass the examination with an average score of 6-7, demonstrating the technology's potential to revolutionize the educational landscape. The analysis of ChatGPT performance revealed its proficiency in a range of subjects, including mathematics, English, physics, chemistry, biology, history, geography, civic education, and literature, which suggests its potential to provide effective support for learners. However, further research is needed to assess ChatGPT performance on more complex exam questions and its potential to support learners in different contexts. As technology continues to evolve and improve, we can expect to see the use of AI tools like ChatGPT become increasingly common in educational settings, ultimately enhancing the educational experience for both students and educators.

Motivation & Objective

• Motivate AI in education by evaluating ChatGPT on a high-stakes exam (VNHSGE).
• Assess ChatGPT's ability to handle multiple subjects using a Vietnamese dataset.
• Compare ChatGPT performance to human student distributions and prior OpenAI-exam results.
• Discuss implications for AI-supported learning and assessment in education.

Proposed method

• Use the VNHSGE evaluation set (multiple subjects: math, English, physics, chemistry, biology, history, geography, civic education, literature).
• Apply zero-shot prompting with a structured answer format to ChatGPT via OpenAI API.
• Grade responses with a binary correctness metric against human ground-truth solutions.
• Compute General Admission Scores (GAS) using the Ministry of Education formula and compare across years 2019–2023.
• Compare subject-by-subject performance to Vietnamese student distributions and to other AI-evaluation cases.

Experimental results

Research questions

• RQ1 Can ChatGPT pass the Vietnamese National High School Graduation Examination (VNHSGE)?
• RQ2 How does ChatGPT's performance vary across different subjects within the VNHSGE?
• RQ3 How does ChatGPT's performance on VNHSGE compare to its performance on other standardized exams and to Vietnamese student score distributions?

Key findings

• ChatGPT achieved an average score range across years and subjects that suggests passing potential (overall GAS scores 6.35–6.94 for natural and social classes).
• Across subjects 2019–2023, ChatGPT's average scores ranged from 4.8 to 7.92, with English generally highest and chemistry generally lowest.
• ChatGPT English performance consistently high (average 7.6–8.6) relative to many Vietnamese students.
• Math, physics, biology, history, geography, and literature showed varying but generally competitive results; chemistry lagged behind in several years.
• Compared to other exams, ChatGPT's performance on VNHSGE was comparable to or better in some areas (e.g., English, literature) but could be weaker in math and chemistry versus some benchmarks; geography limited by lack of image/chart analysis in GPT-3.5.
• Overall, ChatGPT passes the VNHSGE according to the scoring formula, indicating potential as an educational tool and prompt-based exam assistant.