[Paper Review] Impact of Legal Requirements on Explainability in Machine Learning

Adrien Bibal, Michaël Lognoul+2 more|arXiv (Cornell University)|Jul 10, 2020

Explainable Artificial Intelligence (XAI)9 references4 citations

TL;DR

This paper analyzes how European legal frameworks—such as GDPR, consumer protection laws, and financial regulations—impose varying levels of explainability on machine learning models in both private and public decision-making. It maps legal requirements to four tiers of explainability (main features, all features, feature combination, and full model), showing that only interpretable models can satisfy the strictest legal standards, while black-box models face fundamental limitations in compliance.

ABSTRACT

The requirements on explainability imposed by European laws and their implications for machine learning (ML) models are not always clear. In that perspective, our research analyzes explanation obligations imposed for private and public decision-making, and how they can be implemented by machine learning techniques.

Motivation & Objective

• To clarify the legal obligations for explainability in machine learning as defined by European regulations.
• To identify and categorize the varying levels of explainability required by different legal texts in private and public decision-making contexts.
• To assess the feasibility of meeting these legal requirements using existing machine learning techniques.
• To highlight the gap between legal expectations and the capabilities of black-box models in public sector applications.
• To call for deeper collaboration between legal scholars and AI researchers to align technical explainability with legal standards.

Proposed method

• Systematic analysis of European legal texts, including GDPR, Directive 2011/83, Regulation 2019/1150, and Directive 2014/65, to extract explainability requirements.
• Classification of legal requirements into four hierarchical tiers of explainability: main features, all features, feature combination, and full model.
• Mapping of these tiers to machine learning techniques: interpretable models (e.g., linear models, decision trees), feature importance methods (e.g., LIME, permutation importance), and sparsity techniques (e.g., Lasso).
• Evaluation of the limitations of black-box models (e.g., neural networks) in satisfying legal requirements, especially for public decisions.
• Review of existing AI methods for legal reasoning, such as sequence-to-sequence learning and multi-task learning, to address judicial explanation needs.
• Comparison of administrative vs. judicial decision-making requirements, emphasizing the need for factual and legal grounds and responses to legal arguments.

Experimental results

Research questions

• RQ1 What are the specific explainability requirements imposed by European legal frameworks on machine learning systems in private and public decision-making?
• RQ2 How do different legal texts define the scope of required explanations—e.g., focusing on features, combinations, or the full model?
• RQ3 To what extent can interpretable machine learning models and post-hoc explanation techniques satisfy legal requirements for transparency?
• RQ4 Why do black-box models fail to meet the strongest legal standards, particularly in public sector applications?
• RQ5 What are the key challenges in aligning machine learning explainability with legal reasoning, such as motivation and argument response in judicial decisions?

Key findings

• Legal requirements for explainability in machine learning vary significantly across EU regulations, with GDPR, consumer law, and financial regulations imposing distinct tiers of transparency.
• The weakest requirement—providing main features—is achievable with feature importance techniques in black-box models, such as permutation-based importance or out-of-bag error in random forests.
• The requirement to disclose all features used in a decision is challenging for high-dimensional models, necessitating techniques like Lasso to enforce sparsity.
• Explaining how features are combined to produce a decision is feasible with local explanation methods like LIME, but not globally interpretable for complex models.
• The strongest requirement—providing the full model—is only satisfied by inherently interpretable models, as black-box models (e.g., deep neural networks) cannot be fully disclosed or understood.
• Public decision-making, especially in judicial contexts, demands explanation of factual and legal grounds and responses to legal arguments—capabilities currently beyond the reach of standard ML systems.