[论文解读] Towards best practices in AGI safety and governance: A survey of expert opinion

Jonas Schuett, Noemi Dreksler+5 more|arXiv (Cornell University)|May 11, 2023

Ethics and Social Impacts of AI被引用 15

一句话总结

对跨越 AGI 实验室、学术界和民间社会的 51 位专家的调查发现，普遍一致认为 AGI 实验室应采用广泛的安全与治理实践，尤其强烈支持部署前风险评估、危险能力评估、第三方模型审计、安全限制和红队演练。

ABSTRACT

A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve or exceed human performance across a wide range of cognitive tasks. In pursuing this goal, they may develop and deploy AI systems that pose particularly significant risks. While they have already taken some measures to mitigate these risks, best practices have not yet emerged. To support the identification of best practices, we sent a survey to 92 leading experts from AGI labs, academia, and civil society and received 51 responses. Participants were asked how much they agreed with 50 statements about what AGI labs should do. Our main finding is that participants, on average, agreed with all of them. Many statements received extremely high levels of agreement. For example, 98% of respondents somewhat or strongly agreed that AGI labs should conduct pre-deployment risk assessments, dangerous capabilities evaluations, third-party model audits, safety restrictions on model usage, and red teaming. Ultimately, our list of statements may serve as a helpful foundation for efforts to develop best practices, standards, and regulations for AGI labs.

研究动机与目标

• 确定哪些 AGI 实验室的安全与治理实践得到广泛专家支持。
• 评估支持是否因部门（AGI 实验室、学术界、民间社会）或性别而异。
• 为制定 AGI 安全标准、法规与最佳实践奠定基础。
• 向决策者和制定标准的机构传达广泛认可的治理措施。

提出的方法

• 向 92 位受邀专家调查关于 AGI 实验室实践的 50 项陈述；收回 51 份答卷（回复率 55.4%）。
• 5 点李克特量表（-2 到 2）再加上 'I don't know'；30 项为必答，20 项为可选。
• 统计检验：按部门的总体同意程度使用 Mann-Whitney U 检验；逐项差异使用卡方检验；多重检验采用 Holm-Bonferroni 校正。
• 开放科学实践：事前注册、事前分析计划，以及 OSF 数据/代码共享；对人口统计信息进行匿名化报告。

Figure 1: Sample by sector and gender | The figure shows the sector of work and gender of the respondents. Respondents could choose more than one sector in which they work.

实验结果

研究问题

• RQ1 领先专家就 AGI 实验室应实施哪些安全与治理实践达成广泛共识？
• RQ2 不同部门（AGI 实验室、学术界、民间社会）或性别之间的一致性水平是否存在差异？
• RQ3 专家在调查的 50 项之外还提出了哪些额外实践？
• RQ4 这些发现如何为 AGI 安全的政策、标准和监管工作提供依据？

主要发现

• 普遍一致认为 50 项实践中的大多数应予以实施；平均有 85.2% 的受访者对每项实践表示同意。
• 98% 的受访者对核心实践（部署前风险评估、危险能力评估、第三方模型审计、安全限制和红队演练）应予以实施表示或多或少同意。
• 对所有项目的平均同意度为 1.39，量表为 -2 到 2，表明总体偏向同意。
• 来自 AGI 实验室的受访者总体同意程度高于学术界或民间社会，尽管逐项差异并不显著。
• 有 5 项未出现分歧，包括危险能力评估、部署前风险评估，以及发布对齐策略。
• 关于企业风险管理以及某些协调实践（如跨实验室审查、通知其他实验室）存在一些不确定性。
• 受访者提出了除调查清单之外的额外 50 项实践，显示治理设计仍有扩展空间。

Figure 2: Percentages of responses for all statements | The figure shows the percentage of respondents choosing each answer option. At the end of each bar we show the number of people who answered each item. The items are ordered by the total number of respondents that “strongly” agreed. The full st