[论文解读] The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence

Matt White, Ibrahim Haddad+5 more|arXiv (Cornell University)|Mar 20, 2024

Semantic Web and Ontologies被引用 13

一句话总结

本文提出 Model Openness Framework (MOF)，一个三层分类体系以及 Model Openness Tool (MOT)，用于评估并提升 AI 模型在代码、数据与文档方面的开放性、完整性和可重用性。

ABSTRACT

Generative artificial intelligence (AI) offers numerous opportunities for research and innovation, but its commercialization has raised concerns about the transparency and safety of frontier AI models. Most models lack the necessary components for full understanding, auditing, and reproducibility, and some model producers use restrictive licenses whilst claiming that their models are "open source". To address these concerns, we introduce the Model Openness Framework (MOF), a three-tiered ranked classification system that rates machine learning models based on their completeness and openness, following open science principles. For each MOF class, we specify code, data, and documentation components of the model development lifecycle that must be released and under which open licenses. In addition, the Model Openness Tool (MOT) provides a user-friendly reference implementation to evaluate the openness and completeness of models against the MOF classification system. Together, the MOF and MOT provide timely practical guidance for (i) model producers to enhance the openness and completeness of their publicly-released models, and (ii) model consumers to identify open models and their constituent components that can be permissively used, studied, modified, and redistributed. Through the MOF, we seek to establish completeness and openness as core tenets of responsible AI research and development, and to promote best practices in the burgeoning open AI ecosystem.

研究动机与目标

• 在商业化担忧之中，推动前沿 AI 模型实现更高的透明性与可重复性。
• 提出一个三层 MOF 分类，用以按完整性与开放性对模型进行分级。
• 为每个 MOF 类定义所需发布的组件（代码、数据、文档）及相应的许可要求。
• 提供一个参考实现（MOT），以根据 MOF 分类评估开放性。
• 为模型生产者和使用者在开放性与再使用方面提供实际指南。

提出的方法

• 定义一个三层 MOF 分类，按照开放科学原则对模型在完整性与开放性进行评定。
• 为每个 MOF 类规定所需的代码、数据和文档组件及可允许的许可。
• 开发 Model Openness Tool (MOT)，作为用户友好的参考实现，以根据 MOF 标准评估模型的开放性。

实验结果

研究问题

• RQ1 如何在 AI 模型中系统地对开放性与完整性进行分类？
• RQ2 每个 MOF 类需要哪些代码、数据和文档组件，以及应以哪些许可发布？
• RQ3 工具（MOT）如何帮助生产者与消费者评估和比较模型的开放性？

主要发现

• MOF 提供了一个实用的三层分类，用于对模型的开放性与完整性进行等级评定。
• MOT 提供了一个用户友好的实现，用于根据 MOF 标准评估模型。
• 该框架面向模型生产者（提升开放性）和消费者（识别可用的开源模型）。
• MOF 强调在开放许可证下发布组件（代码、数据、文档），以符合开放科学原则。