[Paper Review] An evidence-based methodology for human rights impact assessment (HRIA) in the development of AI data-intensive systems

Alessandro Mantelero, Maria Samantha Esposito|arXiv (Cornell University)|Jul 30, 2024

Privacy, Security, and Data Protection3 references66 citations

TL;DR

This paper proposes an evidence-based Human Rights Impact Assessment (HRIA) methodology for AI data-intensive systems, grounded in empirical analysis of 700+ decisions from European data protection authorities. The model enables measurable, risk-tiered evaluation of AI impacts on human rights, supporting compliance, design iteration, and stakeholder engagement across AI lifecycles.

ABSTRACT

Different approaches have been adopted in addressing the challenges of Artificial Intelligence (AI), some centred on personal data and others on ethics, respectively narrowing and broadening the scope of AI regulation. This contribution aims to demonstrate that a third way is possible, starting from the acknowledgement of the role that human rights can play in regulating the impact of data-intensive systems. The focus on human rights is neither a paradigm shift nor a mere theoretical exercise. Through the analysis of more than 700 decisions and documents of the data protection authorities of six countries, we show that human rights already underpin the decisions in the field of data use. Based on empirical analysis of this evidence, this work presents a methodology and a model for a Human Rights Impact Assessment (HRIA). The methodology and related assessment model are focused on AI applications, whose nature and scale require a proper contextualisation of HRIA methodology. Moreover, the proposed models provide a more measurable approach to risk assessment which is consistent with the regulatory proposals centred on risk thresholds. The proposed methodology is tested in concrete case-studies to prove its feasibility and effectiveness. The overall goal is to respond to the growing interest in HRIA, moving from a mere theoretical debate to a concrete and context-specific implementation in the field of data-intensive applications based on AI.

Motivation & Objective

• To develop a practical, context-specific HRIA methodology for AI systems that moves beyond theoretical ethics toward legal and regulatory implementation.
• To address the limitations of broad data protection regulations in capturing AI-specific human rights risks.
• To provide a measurable, risk-graded assessment tool aligned with emerging regulatory frameworks like the EU AI Act.
• To support both developers and regulators in proactively identifying, assessing, and mitigating human rights risks in AI systems.
• To enable transparent, participatory, and auditable AI development through structured impact reporting and lifecycle monitoring.

Proposed method

• The methodology is built on empirical analysis of 700+ decisions and documents from data protection authorities in six European countries.
• It identifies and maps specific human rights and freedoms impacted by data-intensive AI systems based on real legal and regulatory outcomes.
• The HRIA model integrates risk assessment with measurable indicators for likelihood, severity, and extent of impact, enabling risk tiering.
• The model supports iterative design by allowing comparison between alternative AI configurations and mitigation strategies.
• It includes a HRIA management plan with timelines, responsibilities, and performance indicators for implementation and monitoring.
• The model is tested in concrete case studies and designed for scalability across AI projects, including large-scale systems like smart cities.

Experimental results

Research questions

• RQ1 How do data-intensive AI systems impact fundamental human rights according to real-world regulatory decisions?
• RQ2 What are the key human rights and freedoms most frequently affected by AI applications in practice?
• RQ3 How can HRIA be formalized to provide measurable, risk-graded assessments that align with regulatory thresholds?
• RQ4 In what ways can HRIA support iterative, human-centric AI design and stakeholder participation?
• RQ5 How can HRIA be adapted for use across different legal cultures and integrated into AI lifecycle management?

Key findings

• The empirical analysis of 700+ decisions revealed that human rights, particularly privacy, non-discrimination, and fair treatment, are already central to data protection decisions in Europe.
• Traditional HRIA reports often lack quantitative risk assessment, leaving impact grading and mitigation to duty bearers; the proposed model fills this gap with standardized metrics.
• The HRIA model enables risk tiering, which can prevent high-risk AI systems from entering the market by providing clear regulatory compliance pathways.
• The model supports lifecycle monitoring through periodic audits, progress reports, and human rights indicators, enhancing accountability.
• Integrated assessments are necessary for large-scale AI systems like smart cities, requiring external advisors and co-design processes to capture cumulative socio-technical impacts.
• The model facilitates stakeholder engagement by providing clear, structured, and comparable evidence on AI design choices and their human rights implications.