[Paper Review] Open Problems in Technical AI Governance

Anka Reuel, Ben Bucknall+31 more|arXiv (Cornell University)|Jul 20, 2024

Big Data and Business Intelligence5 citations

TL;DR

This paper defines technical AI governance (TAIG), presents a two-dimensional taxonomy of capacities and targets, and catalogs open technical problems to guide researchers and funders.

ABSTRACT

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

Motivation & Objective

• Motivate the field of technical AI governance and justify its importance.
• Introduce a two-dimensional taxonomy of TAIG (capacities and targets) to structure problems.
• Catalog open technical problems and provide concrete research questions for future work.
• Relate TAIG to broader AI governance literature and policy contexts.

Proposed method

• Define TAIG and articulate its relationship to AI governance.
• Develop a two-axis taxonomy with capacities (e.g., Assessment, Access, Verification, Security, Operationalization, Ecosystem Monitoring) and targets (Data, Compute, Models and Algorithms, Deployment).
• Map open problems to capacity–target pairs and provide example research questions for each area.
• Highlight scope, limitations, and potential dual-use concerns to guide responsible research and funding.
• Offer a reader-guided structure with self-contained sections and a policy brief appendix.

Experimental results

Research questions

• RQ1 How can scalable methods identify problematic data in datasets of trillions of tokens/samples (3.1.1)?
• RQ2 How can automated license collection and metadata reporting prevent training on unlicensed data (3.1.1, 3.1.2)?
• RQ3 How can model behavior be attributed to training data, including pretraining versus fine-tuning data (3.1.3)?
• RQ4 What infrastructure and metadata are needed to analyze large datasets effectively (3.1.2)?
• RQ5 How should compute and hardware specifications be defined and monitored to govern large-scale model training (3.2.1, 3.2.2)?

Key findings

• Introduces the field of technical AI governance (TAIG) and motivates its importance.
• Provides a structured taxonomy of TAIG with capacities and targets.
• Outlines open problems across data, compute, models, deployment, assessment, security, and ecosystem monitoring.
• Offers concrete example research questions and a policy brief to guide technical researchers and funders.
• Emphasizes the need for scalable, robust evaluation methods and governance-oriented tooling.
• Notes dual-use considerations and non-technical governance dimensions.