[Paper Review] The risks of risk-based AI regulation: taking liability seriously

Martin Kretschmer, Tobias Kretschmer+2 more|arXiv (Cornell University)|Nov 3, 2023

Ethics and Social Impacts of AI4 citations

TL;DR

This paper critiques the EU's risk-based AI Act for overrelying on ex-ante regulatory mandates like data quality and human oversight, arguing instead for a liability-driven approach that holds developers and deployers accountable only after harm occurs. By differentiating between endogenous and exogenous sources of risk and allocating liability accordingly, the authors propose that liability enforcement would better incentivize compliance, transparency, and system retraining than prescriptive regulation.

ABSTRACT

The development and regulation of multi-purpose, large "foundation models" of AI seems to have reached a critical stage, with major investments and new applications announced every other day. Some experts are calling for a moratorium on the training of AI systems more powerful than GPT-4. Legislators globally compete to set the blueprint for a new regulatory regime. This paper analyses the most advanced legal proposal, the European Union's AI Act currently in the stage of final "trilogue" negotiations between the EU institutions. This legislation will likely have extra-territorial implications, sometimes called "the Brussels effect". It also constitutes a radical departure from conventional information and communications technology policy by regulating AI ex-ante through a risk-based approach that seeks to prevent certain harmful outcomes based on product safety principles. We offer a review and critique, specifically discussing the AI Act's problematic obligations regarding data quality and human oversight. Our proposal is to take liability seriously as the key regulatory mechanism. This signals to industry that if a breach of law occurs, firms are required to know in particular what their inputs were and how to retrain the system to remedy the breach. Moreover, we suggest differentiating between endogenous and exogenous sources of potential harm, which can be mitigated by carefully allocating liability between developers and deployers of AI technology.

Motivation & Objective

• To challenge the EU’s risk-based AI regulation model, particularly the AI Act’s ex-ante mandates on data quality and human oversight.
• To argue that liability—rather than pre-emptive rules—should be the central regulatory mechanism for AI systems.
• To differentiate between endogenous (systemic) and exogenous (external) sources of AI harm for more precise regulatory allocation.
• To propose that liability enforcement would better incentivize developers to know inputs and retrain systems after breaches.
• To advocate for a shift from prescriptive regulation to outcome-based accountability through civil liability

Proposed method

• Analyzing the EU AI Act’s risk-based framework and its obligations on data quality and human oversight.
• Comparing the EU’s ex-ante regulatory model with traditional ex-post liability models in digital technology.
• Proposing a liability-based regulatory model where firms are held accountable only after a breach occurs.
• Introducing a distinction between endogenous (development-related) and exogenous (deployment/environment-related) sources of AI harm.
• Recommending that liability be allocated based on the source of risk to ensure appropriate incentives and accountability.
• Drawing on existing legal frameworks such as product liability and notice-and-takedown procedures to inform the proposed liability model.

Figure 1: The EU’s risk-based approach to AI regulation

Experimental results

Research questions

• RQ1 How does the EU AI Act’s risk-based, ex-ante regulatory model compare to traditional ex-post liability models in digital technology?
• RQ2 What are the unintended consequences of mandating data quality and human oversight in AI systems under the AI Act?
• RQ3 Why is liability a more effective regulatory mechanism than prescriptive rules for AI development and deployment?
• RQ4 How can liability be differentiated based on whether harm arises from endogenous (systemic) or exogenous (external) sources?
• RQ5 What role should liability play in ensuring developers can identify and retrain systems after a breach?

Key findings

• The EU AI Act’s risk-based approach, particularly its mandates on data quality and human oversight, may impose excessive compliance burdens without ensuring meaningful risk reduction.
• Ex-ante regulation of AI through product safety principles diverges from long-standing digital policy norms that rely on ex-post liability, which has historically supported innovation.
• Liability, when properly structured, provides stronger incentives for developers to track inputs and retrain systems after a breach than prescriptive rules.
• Differentiating between endogenous and exogenous sources of harm allows for more precise allocation of responsibility between developers and deployers.
• The current AI Act framework risks undermining innovation and transparency by prioritizing regulatory control over accountability mechanisms.
• A liability-based model would better align with existing legal principles and improve system-level accountability without overburdening developers with pre-emptive compliance obligations.