[Paper Review] ChatGPT in education: A discourse analysis of worries and concerns on social media

Lingyao Li, Zihui Ma+4 more|arXiv (Cornell University)|Apr 29, 2023

Artificial Intelligence in Healthcare and Education26 citations

TL;DR

The paper analyzes Twitter discourse to identify concerns about ChatGPT in education, using BERT-based topic modeling and social network analysis to map who drives the conversation and what issues dominate.

ABSTRACT

The rapid advancements in generative AI models present new opportunities in the education sector. However, it is imperative to acknowledge and address the potential risks and concerns that may arise with their use. We analyzed Twitter data to identify key concerns related to the use of ChatGPT in education. We employed BERT-based topic modeling to conduct a discourse analysis and social network analysis to identify influential users in the conversation. While Twitter users generally ex-pressed a positive attitude towards the use of ChatGPT, their concerns converged to five specific categories: academic integrity, impact on learning outcomes and skill development, limitation of capabilities, policy and social concerns, and workforce challenges. We also found that users from the tech, education, and media fields were often implicated in the conversation, while education and tech individual users led the discussion of concerns. Based on these findings, the study provides several implications for policymakers, tech companies and individuals, educators, and media agencies. In summary, our study underscores the importance of responsible and ethical use of AI in education and highlights the need for collaboration among stakeholders to regulate AI policy.

Motivation & Objective

• Identify key concerns about using ChatGPT in education from social media discourse.
• Characterize the sentiment and framing of these concerns on Twitter.
• Identify influential users and communities shaping the conversation.
• Discuss policy, implementation, and ethical implications for stakeholders.
• Provide actionable insights for policymakers, tech companies, educators, and media agencies.

Proposed method

• Analyze Twitter data related to ChatGPT in education.
• Apply BERT-based topic modeling to extract discourse topics.
• Perform social network analysis to identify influential users and communities.
• Map concerns into thematic categories.
• Synthesize implications for policymakers, industry, and educators.

Experimental results

Research questions

• RQ1 What are the predominant concerns about ChatGPT in education on Twitter?
• RQ2 Which themes or topics commonly appear in discourse about ChatGPT in education?
• RQ3 Who are the influential actors shaping the conversation on social media?
• RQ4 What implications do these discourse patterns have for policy, practice, and ethics?

Key findings

• Users generally express a positive attitude toward ChatGPT in education.
• Concerns converge into five categories: academic integrity, impact on learning outcomes and skill development, limitation of capabilities, policy and social concerns, and workforce challenges.
• Tech, education, and media fields are often implicated in the conversation, with education and tech users leading the discussion of concerns.
• The study provides implications for policymakers, tech companies, educators, and media agencies.
• It highlights the need for responsible and ethical use of AI in education and collaboration among stakeholders to regulate AI policy.