[Paper Review] Confidence-Building Measures for Artificial Intelligence: Workshop Proceedings

Sarah Shoker, Andrew W. Reddie+21 more|arXiv (Cornell University)|Aug 1, 2023

Scientific Computing and Data Management13 citations

TL;DR

The workshop identifies practical confidence-building measures to mitigate security risks from foundation models, emphasizing multi-stakeholder involvement and adaptable, non-binding actions.

ABSTRACT

Foundation models could eventually introduce several pathways for undermining state security: accidents, inadvertent escalation, unintentional conflict, the proliferation of weapons, and the interference with human diplomacy are just a few on a long list. The Confidence-Building Measures for Artificial Intelligence workshop hosted by the Geopolitics Team at OpenAI and the Berkeley Risk and Security Lab at the University of California brought together a multistakeholder group to think through the tools and strategies to mitigate the potential risks introduced by foundation models to international security. Originating in the Cold War, confidence-building measures (CBMs) are actions that reduce hostility, prevent conflict escalation, and improve trust between parties. The flexibility of CBMs make them a key instrument for navigating the rapid changes in the foundation model landscape. Participants identified the following CBMs that directly apply to foundation models and which are further explained in this conference proceedings: 1. crisis hotlines 2. incident sharing 3. model, transparency, and system cards 4. content provenance and watermarks 5. collaborative red teaming and table-top exercises and 6. dataset and evaluation sharing. Because most foundation model developers are non-government entities, many CBMs will need to involve a wider stakeholder community. These measures can be implemented either by AI labs or by relevant government actors.

Motivation & Objective

• Motivate the need for confidence-building measures (CBMs) in the foundation model era to prevent misperception and escalation.
• Identify a set of actionable CBMs applicable to AI systems across actors (labs, governments, civil society).
• Explain how CBMs can operate alongside formal regulation to manage rapid AI innovation.
• Highlight political and technical limitations that could affect CBM success and adoption.
• Propose pathways for integrating CBMs into existing AI governance frameworks.

Proposed method

• Identify CBMs that apply to foundation models, including crisis hotlines, incident sharing, model/system cards, content provenance and watermarks, collaborative red teaming, tabletop exercises, and data/evaluation sharing.
• Organize CBMs into four categories: communication and coordination, observation and verification, cooperation and integration, and transparency.
• Discuss the role of non-government actors and multistakeholder involvement in implementing CBMs.
• Provide examples and considerations from historical and contemporary international security contexts.
• Assess limitations and the need for ongoing red-teaming and governance alignment.

Experimental results

Research questions

• RQ1 What CBMs are most applicable to mitigating international security risks from foundation models?
• RQ2 How can CBMs be implemented given that many AI developers are non-governmental actors and require multi-stakeholder participation?
• RQ3 What are the political and technical limitations affecting the feasibility and effectiveness of CBMs for AI?
• RQ4 How can CBMs complement existing international regulatory discussions and frameworks?

Key findings

• CBMs identified as applicable to foundation models include crisis hotlines, incident sharing, model/transparency/system cards, content provenance and watermarks, collaborative red teaming, tabletop exercises, and dataset/evaluation sharing.
• CBMs are categorized into communication/coordination, observation/verification, cooperation/integration, and transparency, and are designed to reduce misperception and escalation.
• Many CBMs are voluntary and can be implemented by AI labs or government actors, with potential multi-stakeholder involvement due to the non-government nature of many developers.
• There are political and technical limitations to CBMs, such as verification challenges, incentive alignment, and the evolving nature of AI capabilities requiring adaptable, build-as-you-go approaches.
• Proposed CBMs can complement but not replace formal regulatory efforts, and may serve as a bridge in low-trust international environments.