[论文解读] Did the Roll-Out of Community Notes Reduce Engagement With Misinformation on X/Twitter?

Yuwei Chuai, Haoye Tian+2 more|arXiv (Cornell University)|Jul 16, 2023

Misinformation and Its Impacts参考文献 54被引用 7

一句话总结

这项大规模实证研究评估了X/Twitter的Community Notes——一种众包事实核查功能——是否减少了对虚假信息的互动。基于自2021年以来的全面事实核查与原始推文数据集，采用双重差分法（DiD）和断点回归设计（RDD），研究发现虚假内容的转发量或点赞量均无显著减少，表明Community Notes在影响早期病毒式传播方面反应过慢。

ABSTRACT

Developing interventions that successfully reduce engagement with misinformation on social media is challenging. One intervention that has recently gained great attention is X/Twitter's Community Notes (previously known as "Birdwatch"). Community Notes is a crowdsourced fact-checking approach that allows users to write textual notes to inform others about potentially misleading posts on X/Twitter. Yet, empirical evidence regarding its effectiveness in reducing engagement with misinformation on social media is missing. In this paper, we perform a large-scale empirical study to analyze whether the introduction of the Community Notes feature and its roll-out to users in the U.S. and around the world have reduced engagement with misinformation on X/Twitter in terms of retweet volume and likes. We employ Difference-in-Differences (DiD) models and Regression Discontinuity Design (RDD) to analyze a comprehensive dataset consisting of all fact-checking notes and corresponding source tweets since the launch of Community Notes in early 2021. Although we observe a significant increase in the volume of fact-checks carried out via Community Notes, particularly for tweets from verified users with many followers, we find no evidence that the introduction of Community Notes significantly reduced engagement with misleading tweets on X/Twitter. Rather, our findings suggest that Community Notes might be too slow to effectively reduce engagement with misinformation in the early (and most viral) stage of diffusion. Our work emphasizes the importance of evaluating fact-checking interventions in the field and offers important implications to enhance crowdsourced fact-checking strategies on social media.

研究动机与目标

• 评估X/Twitter上Community Notes的推出是否减少了用户对虚假信息的互动。
• 评估众包事实核查在真实社交媒体环境中的有效性。
• 确定Community Notes是否影响了虚假内容的互动指标（如转发和点赞）。
• 研究事实核查干预措施的时间动态与虚假信息传播速度之间的关系。
• 为社交媒体平台的内容审核策略提供基于证据的见解。

提出的方法

• 采用双重差分法（DiD）模型，比较不同用户群体在Community Notes推出前后互动趋势的变化。
• 应用断点回归设计（RDD），利用事实核查笔记发布时间作为临界点，分析Community Notes对互动的因果影响。
• 使用自2021年初以来所有Community Notes及其对应原始推文的综合数据集，聚焦英文内容。
• 将原始推文分类为三组：F-CRH（被核查为误导性内容）、F-NCRH（未被评为有帮助）和T-NMR（非误导性参考推文），用于对比分析。
• 通过倾向得分匹配筛选对照组，以增强F-CRH与F-NCRH组之间的可比性。
• 基于公开的API数据，分析聚合的互动指标（转发和点赞）作为结果变量。

实验结果

研究问题

• RQ1 Community Notes的引入和推出是否显著减少了X/Twitter上对误导性推文的互动？
• RQ2 Community Notes发布的时间点如何影响其减少虚假信息早期互动的能力？
• RQ3 在不同用户类型（如高粉丝数的认证用户）中，Community Notes的有效性是否存在差异？
• RQ4 与非误导性内容相比，通过Community Notes发布的事实核查在多大程度上影响了用户行为？
• RQ5 Community Notes的响应时间与虚假信息病毒式传播速度相比，如何影响用户互动？

主要发现

• 尽管事实核查数量显著增加，尤其是针对高粉丝量认证用户的推文，但对误导性内容的转发量或点赞量均无统计学上显著减少。
• 研究未发现Community Notes在虚假信息最易传播的早期阶段减少了互动，表明干预存在关键延迟。
• 从误导性推文发布到Community Notes出现的平均时间过长，无法影响互动高峰。
• 事实核查更常应用于高影响力内容，但并未转化为用户互动的可测量减少。
• Community Notes的有效性受限于其响应速度过慢，无法有效遏制虚假信息的快速传播。
• 排除无法获取的推文（占已核查内容的14%）可能导致对功能潜在影响的低估，但并未改变整体的零结果发现。