[Paper Review] Improving Students' Academic Performance with AI and Semantic Technologies

Yixin Cheng|arXiv (Cornell University)|May 2, 2022

Online Learning and Analytics4 citations

TL;DR

This study proposes an AI-driven framework integrating Genetic Algorithm (GA)-optimized Long Short-Term Memory (LSTM) networks with BERT-based semantic embeddings and SemRefD for prerequisite detection to improve student performance prediction and curriculum analysis. It achieves a 2.5% accuracy improvement over prior work in dropout prediction and successfully identifies strong prerequisite relationships between programming courses using semantic similarity and concept-level dependency scoring.

ABSTRACT

Artificial intelligence and semantic technologies are evolving and have been applied in various research areas, including the education domain. Higher Education institutions strive to improve students' academic performance. Early intervention to at-risk students and a reasonable curriculum is vital for students' success. Prior research opted for deploying traditional machine learning models to predict students' performance. In terms of curriculum semantic analysis, after conducting a comprehensive systematic review regarding the use of semantic technologies in the Computer Science curriculum, a major finding of the study is that technologies used to measure similarity have limitations in terms of accuracy and ambiguity in the representation of concepts, courses, etc. To fill these gaps, in this study, three implementations were developed, that is, to predict students' performance using marks from the previous semester, to model a course representation in a semantic way and compute the similarity, and to identify the prerequisite between two similar courses. Regarding performance prediction, we used the combination of Genetic Algorithm and Long-Short Term Memory (LSTM) on a dataset from a Brazilian university containing 248730 records. As for similarity measurement, we deployed BERT to encode the sentences and used cosine similarity to obtain the distance between courses. With respect to prerequisite identification, TextRazor was applied to extract concepts from course description, followed by employing SemRefD to measure the degree of prerequisite between two concepts. The outcomes of this study can be summarized as: (i) a breakthrough result improves Manrique's work by 2.5% in terms of accuracy in dropout prediction; (ii) uncover the similarity between courses based on course description; (iii) identify the prerequisite over three compulsory courses of School of Computing at ANU.

Motivation & Objective

• To improve early detection of at-risk students by enhancing dropout prediction accuracy using hybrid AI models.
• To address limitations in existing course similarity measurement by leveraging BERT for semantic encoding of course descriptions.
• To identify prerequisite relationships between similar courses using concept extraction and dependency scoring.
• To support curriculum design and academic advising through interpretable, AI-powered semantic analysis of university courses.
• To bridge gaps in semantic curriculum analysis by combining deep learning with knowledge representation techniques.

Proposed method

• Employed a Genetic Algorithm (GA) to optimize feature selection for LSTM-based dropout prediction on a Brazilian university dataset with 248,730 records.
• Applied Long Short-Term Memory (LSTM) networks with fully connected layers to model sequential academic performance and predict dropout risk.
• Used BERT-base to embed course descriptions from ANU into dense semantic vectors, enabling cosine similarity computation between courses.
• Implemented TextRazor to extract key concepts from course descriptions, followed by SemRefD to compute prerequisite strength between concept pairs.
• Combined similarity scores and prerequisite metrics to analyze relationships across 1000-, 2000-, and 3000-/4000-level courses in the School of Computing at ANU.
• Validated results using real-world course data, focusing on programming courses like COMP1100, COMP1110, and COMP2100 for case studies.

Experimental results

Research questions

• RQ1 Can a hybrid GA-LSTM model improve the accuracy of student dropout prediction compared to existing methods?
• RQ2 How accurately can BERT-based semantic embeddings measure similarity between university courses based on their descriptions?
• RQ3 To what extent can SemRefD identify meaningful prerequisite relationships between semantically similar courses?
• RQ4 What is the relationship between course similarity and prerequisite strength in computer science curricula?
• RQ5 Can the integration of deep learning and semantic technologies support early intervention and curriculum optimization in higher education?

Key findings

• The GA-LSTM model improved dropout prediction accuracy by 2.5% over Manrique et al.’s best result, achieving a breakthrough in performance on the ARQ dataset.
• BERT-based semantic encoding achieved high cosine similarity scores, with COMP1110 showing the highest average similarity to other 1000-level courses.
• COMP2100 demonstrated a 0.9479 similarity score with COMP1100 and a 0.9543 score with COMP1110, indicating strong semantic alignment.
• The prerequisite score between COMP1100 and COMP2100 reached 13.17, indicating a strong dependency, while COMP1110 and COMP2100 scored 12.06, confirming their prerequisite relationship.
• The relationship between COMP1100 and COMP1110 had a lower prerequisite score of 4.4, suggesting they are more parallel than hierarchical in the curriculum.
• The framework successfully identified that COMP2100 is a key course with strong prerequisites from both COMP1100 and COMP1110, aligning with program structure.