[論文レビュー] "I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data

Mubin Ul Haque, Isuru Dharmadasa+3 more|arXiv (Cornell University)|Dec 12, 2022

Topic Modeling被引用数 210

ひとこと要約

tldr: A mixed-method study of 10,732 English tweets (Dec 5–7, 2022) analyzes topics and sentiments of early ChatGPT adopters on Twitter, revealing predominantly positive outlooks across key topics with some educational and societal concerns.

ABSTRACT

Large language models have recently attracted significant attention due to their impressive performance on a variety of tasks. ChatGPT developed by OpenAI is one such implementation of a large, pre-trained language model that has gained immense popularity among early adopters, where certain users go to the extent of characterizing it as a disruptive technology in many domains. Understanding such early adopters' sentiments is important because it can provide insights into the potential success or failure of the technology, as well as its strengths and weaknesses. In this paper, we conduct a mixed-method study using 10,732 tweets from early ChatGPT users. We first use topic modelling to identify the main topics and then perform an in-depth qualitative sentiment analysis of each topic. Our results show that the majority of the early adopters have expressed overwhelmingly positive sentiments related to topics such as Disruptions to software development, Entertainment and exercising creativity. Only a limited percentage of users expressed concerns about issues such as the potential for misuse of ChatGPT, especially regarding topics such as Impact on educational aspects. We discuss these findings by providing specific examples for each topic and then detail implications related to addressing these concerns for both researchers and users.

研究の動機と目的

• Understand the characteristics of ChatGPT early adopters (location, occupation, verification)
• Identify main topics discussed about ChatGPT on Twitter using topic modeling
• Assess sentiments expressed by early adopters for each identified topic
• Provide implications and future research directions based on sentiments and topics

提案手法

• Dataset construction from Twitter (Dec 5–7, 2022) collecting English tweets containing 'ChatGPT'
• Deduplicate and preprocess (lowercasing, noise removal, stop-word removal, lemmatization)
• Topic modeling with LDA (MALLET) to determine optimal topic count (N=9)
• Manual qualitative sentiment labeling for 9 topic datasets (100 tweets per topic) with -1/0/1 labels
• Open coding to identify patterns within topics; consensus resolution among five authors

実験結果

リサーチクエスチョン

• RQ1 RQ1: What are the characteristics of ChatGPT early adopters?
• RQ2 RQ2: What are the main topics discussed about ChatGPT on Twitter?
• RQ3 RQ3: What sentiments are expressed about ChatGPT topics on Twitter?

主な発見

Sl	Topic Name	Topic keywords
T1	Disruptions to Software Development	code, write, create, program, generate, python, script, developer, error, run
T2	Entertainment and Exercising Creativity	write, story, poem, love, fun, short, joke, style, funny, movie
T3	Natural Language Processing	model, language, generate, data, text, prompt, human, conversation, learn, response
T4	Impact on Educational Aspects	write, student, paper, essay, plan, research, education, school, assignment, teach, homework
T5	Chatbot Intelligence	chatbot, intelligence, artificialintelligence, machinelearning, artificial, user, million, robot, security, app
T6	Impact on Business development	time, startup, business, company, service, true, idea, control, market, customer
T7	Implications for Search Engines	google, search, answer, engine, replace, result, source, StackOverflow, query, internet, avlable
T8	Q&A Testing	question, answer, wrong, test, response, correct, amp, pretty, simple, solve
T9	Future Careers & Opportunities	tool, future, time, people, technology, potential, job, world, change, learn

• Nine topics were identified via LDA: Disruptions to Software Development, Entertainment and Exercising Creativity, Natural Language Processing, Impact on Educational Aspects, Chatbot Intelligence, Impact on Business Development, Implications for Search Engines, Q&A Testing, Future Careers & Opportunities
• Disruptions to software development: 81% positive sentiment
• Entertainment and exercising creativity: 92% positive sentiment
• Natural Language Processing: majority positive with some concerns; 83% positive, 14% negative, 3% neutral
• Chatbot Intelligence: 78% positive, 20% negative, 2% neutral
• Impact on Educational Aspects: 52% positive, 32% negative, 16% neutral
• Impact on Business Development: 75% positive, 5% negative, 20% neutral
• Implications for Search Engines: 54% positive, 15% negative, 31% neutral
• Q&A Testing: 38% positive, 40% neutral, 22% negative
• Future Careers & Opportunities: topic with wide discussion on skills, collaboration, and industry trends