[Paper Review] Exploring User Perspectives on ChatGPT: Applications, Perceptions, and Implications for AI-Integrated Education

Reza Hadi Mogavi, Chao Deng+10 more|arXiv (Cornell University)|May 22, 2023

Artificial Intelligence in Healthcare and Education62 citations

TL;DR

This study analyzes early adopters' experiences and views of ChatGPT in education by performing qualitative content analysis of 6,000 social media texts from Twitter, Reddit, YouTube, and LinkedIn to map educational applications and perceptions.

ABSTRACT

To foster the development of pedagogically potent and ethically sound AI-integrated learning landscapes, it is pivotal to critically explore the perceptions and experiences of the users immersed in these contexts. In this study, we perform a thorough qualitative content analysis across four key social media platforms. Our goal is to understand the user experience (UX) and views of early adopters of ChatGPT across different educational sectors. The results of our research show that ChatGPT is most commonly used in the domains of higher education, K-12 education, and practical skills training. In social media dialogues, the topics most frequently associated with ChatGPT are productivity, efficiency, and ethics. Early adopters' attitudes towards ChatGPT are multifaceted. On one hand, some users view it as a transformative tool capable of amplifying student self-efficacy and learning motivation. On the other hand, there is a degree of apprehension among concerned users. They worry about a potential overdependence on the AI system, which they fear might encourage superficial learning habits and erode students' social and critical thinking skills. This dichotomy of opinions underscores the complexity of Human-AI Interaction in educational contexts. Our investigation adds depth to this ongoing discourse, providing crowd-sourced insights for educators and learners who are considering incorporating ChatGPT or similar generative AI tools into their pedagogical strategies.

Motivation & Objective

• Understand the predominant educational applications of ChatGPT as reported by early adopters.
• Explore initial perceptions of ChatGPT's impact on the educational process (positive, negative, mixed, or neutral).
• Identify stakeholder groups (researchers, educators, learners, parents, general users) contributing to discussions.
• Provide crowd-sourced insights to guide educators and learners considering AI-integrated education.

Proposed method

• Qualitative content analysis of public social media data from Twitter, Reddit, YouTube, and LinkedIn.
• Inclusion criteria requiring posts with ChatGPT and education-related keywords.
• Stratified random sampling of high-visibility posts to reflect diverse early-adopter roles.
• Inductive coding with Braun and Clarke’s six-phase framework to derive themes.
• Inter-coder reliability assessed with Fleiss’ Kappa (κ = 0.87).
• Ethical data collection with API access, anonymization, and institutional approval.

Experimental results

Research questions

• RQ1 RQ1: What are the predominant educational applications of ChatGPT as reported by early adopters?
• RQ2 RQ2: What are the initial perceptions of adopters within the education sector towards ChatGPT, and how do they evaluate its impact on the educational process?

Key findings

• ChatGPT is most frequently used in higher education (24.18%), K-12 education (22.09%), and practical skills learning (15.28%).
• Other notable domains include professional development (9.61%), entrepreneurship education (7.18%), lifelong learning (4.64%), and popular science education (3.25%).
• Discussions on social media center on productivity (73.19%), efficiency (63.02%), and ethics (48.51%).
• Attitudes among early adopters are mixed: 30.62% positive, 17.15% negative, 37.34% mixed, and 14.89% neutral.
• The study analyzed ~150,000 words across 6,000 text samples from Dec 1, 2022 to Apr 22, 2023.