[Paper Review] Governing Through the Cloud: The Intermediary Role of Compute Providers in AI Regulation

Lennart Heim, Tim Fist+6 more|arXiv (Cornell University)|Mar 13, 2024

Blockchain Technology Applications and Security4 citations

TL;DR

This paper proposes that cloud compute providers—such as AWS, Google Cloud, and Microsoft Azure—should serve as intermediaries in AI regulation by leveraging their technical access to infrastructure data to enforce compliance. It outlines four governance roles: securers, record keepers, verifiers, and enforcers, using non-confidential telemetry like compute hours, power draw, and bandwidth to classify workloads and monitor high-risk AI development without compromising privacy.

ABSTRACT

As jurisdictions around the world take their first steps toward regulating the most powerful AI systems, such as the EU AI Act and the US Executive Order 14110, there is a growing need for effective enforcement mechanisms that can verify compliance and respond to violations. We argue that compute providers should have legal obligations and ethical responsibilities associated with AI development and deployment, both to provide secure infrastructure and to serve as intermediaries for AI regulation. Compute providers can play an essential role in a regulatory ecosystem via four key capacities: as securers, safeguarding AI systems and critical infrastructure; as record keepers, enhancing visibility for policymakers; as verifiers of customer activities, ensuring oversight; and as enforcers, taking actions against rule violations. We analyze the technical feasibility of performing these functions in a targeted and privacy-conscious manner and present a range of technical instruments. In particular, we describe how non-confidential information, to which compute providers largely already have access, can provide two key governance-relevant properties of a computational workload: its type-e.g., large-scale training or inference-and the amount of compute it has consumed. Using AI Executive Order 14110 as a case study, we outline how the US is beginning to implement record keeping requirements for compute providers. We also explore how verification and enforcement roles could be added to establish a comprehensive AI compute oversight scheme. We argue that internationalization will be key to effective implementation, and highlight the critical challenge of balancing confidentiality and privacy with risk mitigation as the role of compute providers in AI regulation expands.

Motivation & Objective

• To address the lack of effective enforcement mechanisms in emerging AI regulations like the EU AI Act and US Executive Order 14110.
• To propose compute providers as critical intermediaries in AI governance due to their unique access to infrastructure-level data.
• To identify and analyze four governance capacities—securer, record keeper, verifier, enforcer—that compute providers can fulfill.
• To evaluate the technical feasibility of using non-confidential, already-collected telemetry for regulatory oversight while preserving privacy.
• To explore how international coordination and privacy-preserving techniques can balance risk mitigation with data protection.

Proposed method

• Analyzing existing regulatory frameworks (e.g., US EO 14110) to identify record-keeping mandates for compute providers.
• Mapping observable infrastructure attributes—such as requested hardware configuration, compute hours, power draw, and network bandwidth—to workload classification and resource consumption.
• Assessing the feasibility of using performance counters, memory access patterns, and weight update frequencies to infer model training vs. inference workloads.
• Evaluating privacy-preserving techniques like confidential computing to collect sensitive data (e.g., training datasets, hyperparameters) with customer consent.
• Proposing a tiered oversight model where compute providers report high-level telemetry to regulators while minimizing data exposure.
• Designing a regulatory ecosystem in which compute providers act as intermediaries between regulators and AI developers, reducing compliance burden and increasing enforcement capacity.

Experimental results

Research questions

• RQ1 How can compute providers be legally and ethically integrated into AI regulatory frameworks as intermediaries?
• RQ2 What technical signals derived from infrastructure telemetry can reliably indicate the type and scale of AI workloads (e.g., training vs. inference) without compromising privacy?
• RQ3 To what extent can existing telemetry data—such as compute hours, power draw, and bandwidth—be repurposed for regulatory oversight?
• RQ4 What are the privacy and security trade-offs in extending compute providers’ roles to include verification and enforcement?
• RQ5 How might international coordination and standardized technical instruments enable scalable, privacy-conscious AI governance?

Key findings

• Compute providers already collect non-confidential telemetry such as compute hours, hardware configuration, and power draw, which can be used to classify workload types and estimate resource consumption.
• Network bandwidth utilization between and within AI accelerators correlates strongly with workload characteristics, enabling differentiation between training and inference workloads.
• Memory bandwidth and core utilization patterns can distinguish between training (high, constant) and inference (variable) workloads, offering a technical basis for workload classification.
• Performance counters by numerical precision can differentiate AI workloads from scientific or gaming workloads, providing a direct proxy for computational intensity.
• The modification of model weights in memory is a strong indicator of training activity, and its frequency and magnitude correlate with compute consumption.
• While some signals (e.g., training dataset or hyperparameter access) require new data collection, they can be made privacy-preserving using confidential computing with customer consent.