[Paper Review] Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil society

David Hartmann, José Renato Laranjeira de Pereira+2 more|arXiv (Cornell University)|Feb 26, 2024

Regulation and Compliance Studies7 citations

TL;DR

The paper argues that the EU DSA and AI Act do not yet create a full AI audit ecosystem, highlighting a regulatory gap and advocating for third-party audits and data/model access by researchers and civil society to ensure effective oversight.

ABSTRACT

The European legislature has proposed the Digital Services Act (DSA) and Artificial Intelligence Act (AIA) to regulate platforms and Artificial Intelligence (AI) products. We review to what extent third-party audits are part of both laws and how is access to information on models and the data provided. By considering the value of third-party audits and third-party data access in an audit ecosystem, we identify a regulatory gap in that the AIA does not provide access to data for researchers and civil society. Our contributions to the literature include: (1) Defining an AI audit ecosystem incorporating compliance and oversight. (2) Highlighting a regulatory gap within the DSA and AIA regulatory framework, preventing the establishment of an AI audit ecosystem that has effective oversight by civil society and academia. (3) Emphasizing that third-party audits by research and civil society must be part of that ecosystem, we call for AIA amendments and delegated acts to include data and model access for certain AI products. Furthermore, we call for the DSA to provide NGOs and investigative journalists with data access to platforms by delegated acts and for adaptions and amendments of the AIA to provide third-party audits and data and model access, at least for high-risk systems. Regulations modeled after EU AI regulations should enable data access and third-party audits, fostering an AI audit ecosystem that promotes compliance and oversight mechanisms.

Motivation & Objective

• Define an AI audit ecosystem that accounts for compliance and oversight.
• Identify regulatory gaps in the DSA and AI Act that hinder an AI audit ecosystem.
• Argue for including third-party audits and data/model access for certain AI products in the AI Act.

Proposed method

• Review and synthesize DSA and AI Act provisions related to audits and data access.
• Define audit typologies and auditor roles based on existing literature and regulatory frameworks.
• Analyze case studies of third-party audits and their implications for an EU audit ecosystem.
• Argue for regulatory adaptations to enable data access and third-party audits by researchers and civil society.

Experimental results

Research questions

• RQ1 Does the DSA plus the AI Act suffice to establish an EU AI audit ecosystem?
• RQ2 What role do third-party audits by researchers and civil society play in accountability and oversight?
• RQ3 What data and model access should be provided to enable effective third-party audits for high-risk AI systems?

Key findings

• Third-party audits by researchers and civil society are essential for a functioning AI auditing ecosystem.
• The current DSA and AI Act together do not automatically create an AI audit ecosystem due to gaps in data and model access.
• The AI Act should be adapted to provide data and model access for certain AI products to enable meaningful third-party audits.
• The DSA should enable NGOs and investigative journalists to access platform data via delegated acts.