[论文解读] Excitements and Concerns in the Post-ChatGPT Era: Deciphering Public Perception of AI through Social Media Analysis

Weihong Qi, Jinsheng Pan+2 more|arXiv (Cornell University)|Jul 11, 2023

Misinformation and Its Impacts被引用 10

一句话总结

本文在 Pubic perception of AI on Reddit post-ChatGPT 使用 BERTopic 进行主题建模、零-shot 情感分析，以及 LIWC 基于语言特征，对 388 个 subreddits 和 33,912 条评论进行了分析。

ABSTRACT

As AI systems become increasingly prevalent in various aspects of daily life, gaining a comprehensive understanding of public perception towards these AI systems has become increasingly essential for several reasons such as ethical considerations, user experience, fear, disinformation, regulation, collaboration, and co-creation. In this study, we investigate how mass social media users perceive the recent rise of AI frameworks such as ChatGPT. We collect a total of 33,912 comments in 388 unique subreddits spanning from November 30, 2022 to June 8, 2023 using a list of AI-related keywords. We employ BERTopic to uncover the major themes regarding AI on Reddit. Additionally, we seek to gain deeper insights into public opinion by examining the distribution of topics across different subreddits. We observe that technology-related subreddits predominantly focus on the technical aspects of AI models. On the other hand, non-tech subreddits show greater interest in social issues such as concerns about job replacement or furlough. We leverage zero-shot prompting to analyze the sentiment and perception of AI among individual users. Through a comprehensive sentiment and emotion analysis, we discover that tech-centric communities exhibit greater polarization compared to non-tech communities when discussing AI topics. This research contributes to our broader understanding of public opinion surrounding artificial intelligence.

研究动机与目标

• 了解大众社交媒体用户如何看待像 ChatGPT 这样的 AI 框架的崛起。
• 识别来自多样化 Reddit 社区的主要主题与主题分布。
• 分析跨主题与子版块的情感与情绪语调，以揭示技术导向与非技术群体之间的差异。

提出的方法

• 使用 AI 相关关键词，从 388 个 subreddits 收集 33,912 条 Reddit 评论（2022-11-30 – 2023-06-08）。
• 应用 BERTopic 与 all-MiniLM-L6-v2 嵌入来提取 232 个主题（去除 16 个异常值）。
• 使用零-shot 提示（gpt-3.5-turbo）获取情感标签（积极、中性、消极）并通过基于关键词的聚合来分配最终情感。
• 执行 LIWC 基于语言分析以获得 Tone、Emotion、Prosocial 和 Conflict 指标。
• 运行线性回归比较 tech-centric 与 non-tech 子版块之间的 LIWC 属性（Tech 作为二元变量）。
• 进行跨 subreddits 的分析以绘制话题分布和情感在各社区中的差异。

实验结果

研究问题

• RQ1 RQ1: What topics characterize AI discussions on Reddit and how do they vary across subreddits?
• RQ2 RQ2: What is the prevailing sentiment for the most discussed topics, and do sentiments differ among subreddits?

主要发现

• Top topics include Consciousness and Intelligence of AI, AI in Gaming and Strategy, AI Model and Prompt Engineering, OpenAI and AI Industry, and AI and Job Automation.
• Tech-centric subreddits focus more on technical aspects, while non-tech subreddits emphasize social implications like job replacement and privacy.
• Tech communities exhibit higher positive tone and higher overall emotional expression, indicating greater polarization in sentiment.
• Tech subreddits show higher Prosocial scores, suggesting more signaling of helping/caring in AI discussions.
• Across subreddits, several topics show higher positive sentiment (e.g., AI in Gaming and Strategy, The Potential Impact of AI on Society) while others show more negative sentiment (e.g., Consciousness and Intelligence of AI, AI Industry, Job Automation).
• Cross-subreddit analysis reveals three topics (Consciousness/Intelligence, OpenAI/AI Industry, AI and Job Automation) as particularly prevalent across multiple subreddits.