[论文解读] Introducing v0.5 of the AI Safety Benchmark from MLCommons

Bertie Vidgen, Adarsh Agrawal+98 more|arXiv (Cornell University)|Apr 18, 2024

Adversarial Robustness in Machine Learning被引用 5

一句话总结

该论文介绍 MLCommons 的 AI 安全基准 v0.5，适用于英语语言聊天调优的语言模型，详细说明其 hazards 分类、测试设计、打分以及开放的基准工具，并有 towards v1.0 的发展计划。

ABSTRACT

This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark.

研究动机与目标

• 推动需要 standardized AI 安全评估的重要性，并在行业与研究领域促进安全改进。
• 提供一个用于为聊天调优的语言模型制定和构建安全基准的 principled 框架。
• 定义角色、用例以及一个 13-项 hazard 分类，指导安全测试。
• 提供一个开放、可扩展的平台（ModelBench）和一个透明的评分类法，用于评估被测系统。

提出的方法

• 定义用例（成人对话到通用英文助手）以及三个用户画像（典型成人、恶意意图、有自伤风险）。
• 制定一个 13 类 hazard 分类，v0.5 范围内有七类入选，并提供详细定义及子类别。
• 创建七个测试集（每个在范围内的 hazard 分类一个）由模板构建的测试项（提示）总计 43,090 条。
• 提出一个开放、可解释的评分类法，用于对照基准评估被测系统。
• 发布一个开源评估平台（ModelBench）和 ModelGauge 测试引擎以执行测试。
• 对被测试模型进行匿名化处理，并提供对十几种开放聊天调优语言模型的示例评估；记录局限性和利益相关方反馈流程。

Figure 1: Scores of systems under test (SUTs) across the different hazard categories)within the AI Safety Benchmark v0.5. The bottom row reports the overall benchmark performance (i.e., lowest grade across hazard categories). Reference models for scoring are denoted with a $(*)$ .

实验结果

研究问题

• RQ1 在聊天调优的语言模型中，评估安全风险的实用、可扩展的分类与测试设计是什么？
• RQ2 我们如何构建用例、角色画像和测试项，以获得可重复、可解释的安全评估？
• RQ3 在 AI 安全领域开放基准的局限性与风险有哪些，治理与发布规则如何维护完整性？
• RQ4 模型打分应如何标准化并可针对不同用例进行调整，同时保持透明？

主要发现

• v0.5 范围内包含七个 hazard 分类，总共定义了 13 个分类（七类在 v0.5 范围内，六类推迟到未来版本）。
• 基准由通过模板创建的 43,090 个测试项组成，用于评估被测系统的安全性。
• 一个开放、模块化的评估工具链（ModelBench 和 ModelGauge）支持标准化、版本化的基准运行，并对新系统的扩展性开放。
• 在发布的评估结果中对模型进行了匿名处理，以保护涉及安全的敏感内容，防止对提示的定向利用。
• 论文提供了详细的发布框架，包括开放原则、许可（软件 Apache 2.0，资源 CC-BY）以及为保持基准完整性而设定的严格发布者要求。