[论文解读] "Learn the Facts About COVID-19": Analyzing the Use of Warning Labels on TikTok Videos

Ling Chen, Krishna P. Gummadi+1 more|arXiv (Cornell University)|Jan 19, 2022

Misinformation and Its Impacts被引用 5

一句话总结

本研究分析了疫情期间TikTok在健康类视频上使用“了解关于COVID-19的事实”警告标签的情况，发现这些标签被广泛且常常不恰当地应用——99%的#coronavirus相关视频被贴上标签，23%的无关视频被错误标记。尽管如此，仍有7.7%的虚假信息视频未被贴上标签，暴露出软性内容审核系统存在显著缺陷。

ABSTRACT

During the COVID-19 pandemic, health-related misinformation and harmful content shared online had a significant adverse effect on society. To mitigate this adverse effect, mainstream social media platforms employed soft moderation interventions (i.e., warning labels) on potentially harmful posts. Despite the recent popularity of these moderation interventions, we lack empirical analyses aiming to uncover how these warning labels are used in the wild, particularly during challenging times like the COVID-19 pandemic. In this work, we analyze the use of warning labels on TikTok, focusing on COVID-19 videos. First, we construct a set of 26 COVID-19 related hashtags, then we collect 41K videos that include those hashtags in their description. Second, we perform a quantitative analysis on the entire dataset to understand the use of warning labels on TikTok. Then, we perform an in-depth qualitative study, using thematic analysis, on 222 COVID-19 related videos to assess the content and the connection between the content and the warning labels. Our analysis shows that TikTok broadly applies warning labels on TikTok videos, likely based on hashtags included in the description. More worrying is the addition of COVID-19 warning labels on videos where their actual content is not related to COVID-19 (23% of the cases in a sample of 143 English videos that are not related to COVID-19). Finally, our qualitative analysis on a sample of 222 videos shows that 7.7% of the videos share misinformation/harmful content and do not include warning labels, 37.3% share benign information and include warning labels, and that 35% of the videos that share misinformation/harmful content (and need a warning label) are made for fun. Our study demonstrates the need to develop more accurate and precise soft moderation systems, especially on a platform like TikTok that is extremely popular among people of younger age.

研究动机与目标

• 调查TikTok在疫情期间如何对与COVID-19相关的视频应用警告标签。
• 评估警告标签在虚假信息和无害内容上应用的准确性和一致性。
• 理解内容与标签存在之间的脱节，特别是针对幽默或非虚假信息内容。
• 评估TikTok这类以青少年为中心的平台中软性内容审核系统的有效性和可信度。
• 为平台透明度提供依据，并改进未来针对健康类虚假信息的内容审核策略。

提出的方法

• 通过TikTok的搜索API收集了41,853条使用26个与COVID-19相关的标签的TikTok视频。
• 对完整数据集进行定量分析，根据标签和内容相关性测量标签的普遍性。
• 对随机抽取的222条英文视频进行定性主题分析，以评估内容类型和标签合理性的依据。
• 使用基于标签的启发式方法推断标签应用逻辑，假设标签由视频描述中的特定标签触发。
• 将视频分类为：虚假信息/有害、无害或娱乐向，并与标签存在情况进行交叉比对。
• 测量误报率（无害视频被贴标签）和漏报率（虚假信息未被贴标签），以评估审核准确性。

实验结果

研究问题

• RQ1 含有#coronavirus或其他与COVID-19相关的标签的TikTok视频中，警告标签被应用的频率如何？
• RQ2 警告标签在与COVID-19疫情无关的视频上被错误应用的程度有多大？
• RQ3 含有虚假信息或有害内容的COVID-19相关视频中，有多少比例未被贴上警告标签？
• RQ4 视频的幽默或娱乐性质与其被错误标记或未被标记（但本应被标记）的可能性之间存在何种关系？
• RQ5 对于明确为娱乐制作的视频与教育或信息类内容，其内容类型与标签存在之间的相关性如何？

主要发现

• 99%的在描述中包含#coronavirus标签的视频均被贴上“了解关于COVID-19的事实”警告标签，表明其采用的是广泛依赖标签的审核策略。
• 在143条与COVID-19无关的英文视频中，有23%被错误地贴上“了解关于COVID-19的事实”警告标签，表明误报率极高。
• 7.7%的含有虚假信息或有害内容的COVID-19相关视频未被贴上标签，暴露出严重的漏报问题。
• 37.3%的传播无害信息的视频被错误标记，表明当前系统应用标签过于宽泛，缺乏细致区分。
• 35%的传播虚假信息或有害内容且本应被标记的视频是出于娱乐或幽默目的制作的，表明利用幽默掩盖危险内容存在风险。
• 本研究揭示TikTok当前的软性内容审核系统存在不一致性，警告标签往往仅基于标签而非内容分析被应用，从而削弱了其可信度和有效性。