[Paper Review] Generative AI in Education: A Study of Educators' Awareness, Sentiments, and Influencing Factors

Aashish Ghimire, James Prather+1 more|arXiv (Cornell University)|Mar 22, 2024

Diverse Approaches in Healthcare and Education Studies14 citations

TL;DR

The study surveys university instructors to assess awareness, sentiments, and drivers of attitudes toward Generative AI and LLMs in higher education, revealing overall positive attitudes with discipline and familiarity differences. CS educators show higher technical understanding and sentiment but similar content-detection confidence compared to non-CS peers.

ABSTRACT

The rapid advancement of artificial intelligence (AI) and the expanding integration of large language models (LLMs) have ignited a debate about their application in education. This study delves into university instructors' experiences and attitudes toward AI language models, filling a gap in the literature by analyzing educators' perspectives on AI's role in the classroom and its potential impacts on teaching and learning. The objective of this research is to investigate the level of awareness, overall sentiment towardsadoption, and the factors influencing these attitudes for LLMs and generative AI-based tools in higher education. Data was collected through a survey using a Likert scale, which was complemented by follow-up interviews to gain a more nuanced understanding of the instructors' viewpoints. The collected data was processed using statistical and thematic analysis techniques. Our findings reveal that educators are increasingly aware of and generally positive towards these tools. We find no correlation between teaching style and attitude toward generative AI. Finally, while CS educators show far more confidence in their technical understanding of generative AI tools and more positivity towards them than educators in other fields, they show no more confidence in their ability to detect AI-generated work.

Motivation & Objective

• Assess educators' awareness of Generative AI tools across departments.
• Evaluate educators' sentiments toward adopting AI tools in education.
• Identify factors that influence variations in teachers' attitudes toward generative AI.
• Compare attitudes between Computer Science and non-Computer Science educators.
• Highlight opportunities and concerns reported by educators regarding AI in the classroom.

Proposed method

• Mixed-methods design with a Likert-scale survey and optional follow-up interviews.
• Quantitative analysis using descriptive statistics, inference tests, and regression analysis.
• Grounded theory approach for qualitative interview coding with inter-rater reliability >85%.
• Integration of qualitative and quantitative results to triangulate educators’ attitudes.

Experimental results

Research questions

• RQ1 RQ1 How aware are educators of Generative AI-based tools across various departments?
• RQ2 RQ2 What are educators’ perceptions and sentiments about these AI tools?
• RQ3 RQ3 What factors contribute to variations in teachers’ attitudes toward generative AI based tools?
• RQ4 RQ4 How do the attitudes and perceptions of CS educators differ from those of educators in different departments?
• RQ5 RQ5 What are the biggest opportunities and concerns identified by the educators?

Key findings

• Most educators have heard of or tried Generative AI tools, with over 40% using them periodically or regularly.
• Overall sentiment toward AI tools is positive (mean 3.99; median 4.5; third quartile 5).
• CS educators show higher technical understanding (83% vs 10% non-CS) and higher familiarity (M=4, SD=.71 vs M=3.15, SD=1.08).
• CS instructors are more confident in students' use of tools (M=4.22) than non-CS instructors (M=3.23).
• Top factors shaping sentiment: benefits outweigh risks, perceived educational quality gains, and ease of integration positively influence attitudes; concerns about creativity loss and cheating negatively influence attitudes.
• Regression analyses (Linear, Random Forest, Gradient Boost, XGBoost) yield MSEs between 0.4 and 0.5 for predicting sentiment based on identified features.