[Paper Review] Towards Responsible AI in Banking: Addressing Bias for Fair Decision-Making

Alessandro Castelnovo|arXiv (Cornell University)|Jan 13, 2024

Ethics and Social Impacts of AI4 citations

TL;DR

This PhD thesis proposes a framework for responsible AI in banking by addressing bias in automated decision-making through three pillars: understanding, mitigating, and accounting for bias. Developed in collaboration with Intesa Sanpaolo, the approach integrates fairness, explainability, and human oversight, with open-source tools like Bias On Demand and FairView enabling practical implementation and advancing AI fairness in real-world financial applications.

ABSTRACT

In an era characterized by the pervasive integration of artificial intelligence into decision-making processes across diverse industries, the demand for trust has never been more pronounced. This thesis embarks on a comprehensive exploration of bias and fairness, with a particular emphasis on their ramifications within the banking sector, where AI-driven decisions bear substantial societal consequences. In this context, the seamless integration of fairness, explainability, and human oversight is of utmost importance, culminating in the establishment of what is commonly referred to as "Responsible AI". This emphasizes the critical nature of addressing biases within the development of a corporate culture that aligns seamlessly with both AI regulations and universal human rights standards, particularly in the realm of automated decision-making systems. Nowadays, embedding ethical principles into the development, training, and deployment of AI models is crucial for compliance with forthcoming European regulations and for promoting societal good. This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias. These contributions are validated through their practical application in real-world scenarios, in collaboration with Intesa Sanpaolo. This collaborative effort not only contributes to our understanding of fairness but also provides practical tools for the responsible implementation of AI-based decision-making systems. In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages, further promoting progress in the field of AI fairness.

Motivation & Objective

• To address bias in AI-driven decision-making within the banking sector to ensure fairness and ethical compliance.
• To develop a comprehensive framework integrating fairness, explainability, and human oversight for responsible AI.
• To align AI systems with European regulations and universal human rights standards in automated financial decisions.
• To provide practical, deployable tools for bias detection and mitigation in real-world banking environments.

Proposed method

• The framework is structured around three pillars: understanding bias through data and model analysis, mitigating bias using fairness-aware machine learning techniques, and accounting for bias via continuous monitoring and audit trails.
• The approach integrates explainability methods to enhance transparency of AI decisions, ensuring stakeholders can interpret and challenge outcomes.
• Collaboration with Intesa Sanpaolo enabled real-world validation of the framework on actual banking datasets and workflows.
• Open-source Python packages—Bias On Demand and FairView—were developed to support bias detection, fairness assessment, and model interpretability.
• The methodology emphasizes iterative feedback loops between model development, fairness evaluation, and human-in-the-loop oversight.
• The framework supports compliance with upcoming European AI regulations by embedding ethical principles throughout the AI lifecycle.

Experimental results

Research questions

• RQ1 How can bias in AI-driven credit scoring and loan approval systems be systematically identified and measured in real banking contexts?
• RQ2 What technical and organizational strategies can effectively mitigate bias while preserving model performance and regulatory compliance?
• RQ3 How can fairness, explainability, and human oversight be meaningfully integrated into the AI development lifecycle in financial institutions?
• RQ4 What open-source tools can empower banks to implement and monitor fairness in AI systems at scale?

Key findings

• The framework successfully identified and mitigated demographic and socioeconomic biases in real-world credit risk models through targeted preprocessing and post-processing techniques.
• The integration of explainability methods improved stakeholder trust and enabled auditable decision-making processes in banking applications.
• The open-source tools Bias On Demand and FairView were adopted in production workflows, demonstrating practical utility and scalability in financial institutions.
• Collaborative deployment with Intesa Sanpaolo validated the framework’s effectiveness in reducing unfair outcomes across diverse customer segments.
• The approach achieved measurable improvements in fairness metrics (e.g., equal opportunity, demographic parity) without significant degradation in predictive performance.
• The study demonstrated that responsible AI is feasible in regulated banking environments when supported by technical tools, governance, and regulatory alignment.