[Paper Review] A literature survey on student feedback assessment tools and their usage in sentiment analysis

Himali Aryal|arXiv (Cornell University)|Sep 9, 2021

Online Learning and Analytics59 references4 citations

TL;DR

This paper surveys feedback assessment tools like Kahoot!, Mentimeter, and Padlet in online education, proposing a sentiment analysis framework to extract student attitudes from qualitative feedback. It identifies Kahoot! as the most widely used tool and highlights a research gap in sentiment analysis of tool-specific feedback, proposing a pipeline involving data collection, preprocessing, feature extraction, and classifier training for sentiment classification.

ABSTRACT

Online learning is becoming increasingly popular, whether for convenience, to accommodate work hours, or simply to have the freedom to study from anywhere. Especially, during the Covid-19 pandemic, it has become the only viable option for learning. The effectiveness of teaching various hard-core programming courses with a mix of theoretical content is determined by the student interaction and responses. In contrast to a digital lecture through Zoom or Teams, a lecturer may rapidly acquire such responses from students' facial expressions, behavior, and attitude in a physical session, even if the listener is largely idle and non-interactive. However, student assessment in virtual learning is a challenging task. Despite the challenges, different technologies are progressively being integrated into teaching environments to boost student engagement and motivation. In this paper, we evaluate the effectiveness of various in-class feedback assessment methods such as Kahoot!, Mentimeter, Padlet, and polling to assist a lecturer in obtaining real-time feedback from students throughout a session and adapting the teaching style accordingly. Furthermore, some of the topics covered by student suggestions include tutor suggestions, enhancing teaching style, course content, and other subjects. Any input gives the instructor valuable insight into how to improve the student's learning experience, however, manually going through all of the qualitative comments and extracting the ideas is tedious. Thus, in this paper, we propose a sentiment analysis model for extracting the explicit suggestions from the students' qualitative feedback comments.

Motivation & Objective

• To evaluate the effectiveness of digital feedback tools like Kahoot!, Mentimeter, and Padlet in online learning environments.
• To investigate the use of sentiment analysis on qualitative feedback collected via these tools to understand student attitudes.
• To identify research gaps in sentiment analysis applied specifically to feedback from assessment tools, as opposed to general platforms like Coursera.
• To propose a structured sentiment analysis pipeline for real-time teaching adaptation based on student feedback.

Proposed method

• Data collection from student feedback via Kahoot!, Mentimeter, and Padlet during a Fall 2021 course offering.
• Text preprocessing including case normalization, stopword removal, punctuation stripping, and handling of exclamation/question marks.
• Feature extraction using term frequency, part-of-speech tagging, negation detection, and integration of lexical resources like SentiWordNet and WordNet for semantic context.
• Model training using multiple classifiers such as Naive Bayes, Support Vector Machines, and Convolutional Neural Networks (CNN).
• Application of ontologies and thesauri to enhance semantic representation and opinion detection in student comments.
• Evaluation of model performance across different feature sets and algorithms to identify the optimal configuration.

Experimental results

Research questions

• RQ1 How many papers were published between 2015 and 2021 using digital tools for feedback collection or quizzes?
• RQ2 In which publications have studies on feedback assessment tools been primarily published?
• RQ3 Which feedback assessment tool has been most frequently explored in research?
• RQ4 When did the majority of research on feedback tools emerge?
• RQ5 What are the most widely used evaluation metrics for feedback technologies in educational research?
• RQ6 How effective is real-time adaptation of lectures based on student feedback during live sessions?
• RQ7 How can sentiment analysis be effectively applied to qualitative feedback from tools like Kahoot!, Mentimeter, and Padlet to assess student attitudes?

Key findings

• Kahoot! was identified as the most popular feedback assessment tool among the surveyed technologies.
• A total of 154 articles were included in the metadata analysis for the literature survey.
• Research on sentiment analysis of feedback from tools like Kahoot!, Mentimeter, and Padlet remains largely unexplored, despite extensive work on sentiment analysis from platforms like Coursera and social media.
• The study found that real-time feedback systems are positively received by both students and instructors, supporting dynamic teaching adaptation.
• The proposed sentiment analysis model includes data preprocessing, semantic feature extraction using lexical databases, and classifier training for improved sentiment classification accuracy.
• There is a significant research gap in applying sentiment analysis specifically to feedback collected from in-class assessment tools, indicating a need for future work in this domain.