[Paper Review] E-cheating Prevention Measures: Detection of Cheating at Online Examinations Using Deep Learning Approach -- A Case Study

Leslie Ching Ow Tiong, Hee-Jeong Lee|arXiv (Cornell University)|Jan 24, 2021

Academic integrity and plagiarism30 citations

TL;DR

The paper proposes an e-cheating intelligent agent with an IP detector and DenseLSTM-based behaviour detector to detect and prevent cheating in online exams, evaluated on a new dataset with high accuracy.

ABSTRACT

This study addresses the current issues in online assessments, which are particularly relevant during the Covid-19 pandemic. Our focus is on academic dishonesty associated with online assessments. We investigated the prevalence of potential e-cheating using a case study and propose preventive measures that could be implemented. We have utilised an e-cheating intelligence agent as a mechanism for detecting the practices of online cheating, which is composed of two major modules: the internet protocol (IP) detector and the behaviour detector. The intelligence agent monitors the behaviour of the students and has the ability to prevent and detect any malicious practices. It can be used to assign randomised multiple-choice questions in a course examination and be integrated with online learning programs to monitor the behaviour of the students. The proposed method was tested on various data sets confirming its effectiveness. The results revealed accuracies of 68% for the deep neural network (DNN); 92% for the long-short term memory (LSTM); 95% for the DenseLSTM; and, 86% for the recurrent neural network (RNN).

Motivation & Objective

• Analyze limitations of online cheating prevention in current online education systems.
• Propose an AI-based e-cheating intelligence agent with network IP detection and behaviour detection modules.
• Create a new dataset for online examination cheating detection (7wiseup).
• Evaluate DenseLSTM against benchmark networks on mid-term and final-term exam data.
• Provide insights for integrating intelligent tutoring system enhancements to mitigate e-cheating.

Proposed method

• Two-stage e-cheating intelligence agent comprising an IP detector and a behaviour detector.
• IP detector randomly assigns question sets and reassigns on detected anomalies based on IP history.
• Behaviour detector uses a DenseLSTM network to classify normal vs abnormal behaviours from one-hot encoded features.
• Data preprocessing converts exam records into one-hot encoded vectors of 23 features (20 question answers plus three speed indicators).
• DenseLSTM architecture includes a conv layer, two LSTM blocks with dense connections, a transition layer, and a softmax loss for classification.
• Training uses softmax cross-entropy loss with Adam optimizer under specified hyperparameters; experiments performed on an RTX 2080 Ti GPU.

Experimental results

Research questions

• RQ1 How effective is the DenseLSTM-based behaviour detector in distinguishing normal versus abnormal behaviour during online exams?
• RQ2 Does the IP-based detector effectively deter or detect suspicious changes in exam conditions (e.g., IP changes, randomised question sets) during online assessments?
• RQ3 How does DenseLSTM compare to DNN, LSTM, and RNN benchmarks in detecting e-cheating across mid-term and final-term datasets?

Key findings

• DenseLSTM achieved the highest overall accuracy of 95.32% across mid-term and final-term examinations.
• DenseLSTM achieved 97.77% accuracy on the mid-term and 92.86% on the final-term datasets.
• LSTM achieved 91.89% overall accuracy, with 94.49% mid-term and 89.29% final-term.
• DNN performed poorly with 67.71% overall accuracy (82.74% mid-term, 52.68% final-term).
• AUC values for DenseLSTM were 0.9972 (mid-term) and 0.9760 (final-term), indicating high discrimination performance.