[Paper Review] Risks and Opportunities of Open-Source Generative AI

Francisco Eiras, Aleksander Petrov+23 more|arXiv (Cornell University)|May 14, 2024

Scientific Computing and Data Management6 citations

TL;DR

The paper argues that open-source generative AI offers net benefits in near to mid-term and long-term horizons, discusses an openness taxonomy for model components, reviews governance in multiple regions, and provides policy and best-practice recommendations to mitigate risks.

ABSTRACT

Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about the potential risks of the technology, and resulted in calls for tighter regulation, in particular from some of the major tech companies who are leading in AI development. This regulation is likely to put at risk the budding field of open-source generative AI. Using a three-stage framework for Gen AI development (near, mid and long-term), we analyze the risks and opportunities of open-source generative AI models with similar capabilities to the ones currently available (near to mid-term) and with greater capabilities (long-term). We argue that, overall, the benefits of open-source Gen AI outweigh its risks. As such, we encourage the open sourcing of models, training and evaluation data, and provide a set of recommendations and best practices for managing risks associated with open-source generative AI.

Motivation & Objective

• Define a three-stage framework (near, mid, long-term) for Gen AI development and assess open-source implications in each stage.
• Propose an openness taxonomy for Gen AI model components and classify prominent LLMs by their level of openness.
• Examine governance, regulatory landscapes, and regional policies affecting open-source Gen AI across the EU, US, China, Middle East, and other regions.
• Identify risks and mitigation strategies for open-source Gen AI and advocate for responsible open sourcing in near to mid-term stages.
• Offer policy recommendations and best practices to balance innovation with safety and accountability.

Proposed method

• Develop a three-stage development framework for Gen AI focusing on adoption rates and capability growth (near, mid, long-term).
• Construct an openness taxonomy distinguishing fully closed, semi-open (with subcategories), and fully open components for code and data in the training, evaluation, and deployment pipeline.
• Apply the taxonomy to 45 high-impact LLMs (2019–2024) to assess openness of pipeline components and model weights, data, and evaluation code.
• Perform a contrastive socio-technical analysis across four impact areas (research/innovation, safety/security, equity/access, societal impacts) for near to mid-term models, and discuss long-term AGI considerations and alignment.
• Survey global regulatory frameworks (EU AI Act, Biden EO, China Generative AI Measures, Middle East policies) and summarize how they affect open-source Gen AI.
• Provide recommendations and best practices for policymakers and developers to enable safe, responsible open sourcing of Gen AI.

Figure 1 : Three Development Stages for Generative AI Models : near-term is defined by early use and exploration of the technology in much of its current state; mid-term is a result of the widespread adoption of the technology and further scaling at current pace; long-term is the result of technolog

Experimental results

Research questions

• RQ1 What is the open-source openness landscape for current Gen AI models across training, evaluation, and deployment components?
• RQ2 What are the near to mid-term risks and opportunities of open-source Gen AI, and how can they be mitigated?
• RQ3 How might long-term developments (e.g., AGI) influence the governance and safety benefits of open sourcing?
• RQ4 What regulatory frameworks exist or are being developed, and how do they shape open-source Gen AI practices?
• RQ5 What policy and operational best practices can maximize benefits while minimizing risks of open-source Gen AI?

Key findings

• Open-source Gen AI offers net positive impacts in near to mid-term for research and innovation, but faces performance gaps relative to closed models.
• There is a skew towards closed components in training data and safety evaluation, which constrains open-source benefits and risk mitigation.
• Models with more open pipelines tend to underperform compared to closed models, indicating a trade-off between openness and performance.
• Regulatory regimes (EU AI Act, US Executive Order, China measures) shape governance but generally recognize economic benefits of open-source AI and emphasize transparency and safety obligations.
• Long-term AGI considerations emphasize technical alignment and risk mitigation through open sourcing as a potential mechanism to reduce existential risk and improve decentralization of governance.
• The paper advocates responsible open-source development with policy guidance and best practices to balance openness with safety.

Figure 2 : Model Pipeline : pipeline of model (1) training, (2) evaluation and (3) deployment analyzed in the report. The component Common Benchmarks Evaluation (in light gray) is included in the pipeline for completeness yet will not be analyzed in detail as these are commonly available and transve