[Paper Review] Joint Optimization of AI Fairness and Utility: A Human-Centered Approach

Yunfeng Zhang, Rachel Bellamy+1 more|arXiv (Cornell University)|Feb 5, 2020

Ethics and Social Impacts of AI16 references32 citations

TL;DR

This paper proposes a human-centered framework for jointly optimizing AI fairness and utility by eliciting policy makers' preferences through the Analytic Hierarchy Process (AHP), then using Bayesian optimization to find model thresholds that balance fairness metrics (statistical parity difference, weighted average odds difference) and utility. The key contribution is a principled, interactive method that enables stakeholders to define trade-offs transparently, resulting in models that reflect real-world fairness priorities while maintaining high utility.

ABSTRACT

Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased and making questionable and unfair decisions. The AI research community has proposed many methods to measure and mitigate unwanted biases, but few of them involve inputs from human policy makers. We argue that because different fairness criteria sometimes cannot be simultaneously satisfied, and because achieving fairness often requires sacrificing other objectives such as model accuracy, it is key to acquire and adhere to human policy makers' preferences on how to make the tradeoff among these objectives. In this paper, we propose a framework and some exemplar methods for eliciting such preferences and for optimizing an AI model according to these preferences.

Motivation & Objective

• To address the gap in existing AI fairness methods that do not incorporate input from human policy makers on trade-offs between fairness and utility.
• To develop a systematic approach for eliciting stakeholder preferences on fairness criteria and model performance.
• To optimize AI models based on these preferences, ensuring fairness and utility are balanced according to stakeholder priorities.
• To create a transparent, interactive system that supports diverse stakeholders—such as judges, defendants, and citizens—in shaping fair AI decisions.

Proposed method

• Uses the Analytic Hierarchy Process (AHP) to elicit policy makers’ relative preferences among fairness metrics (SPD, WAOD) and utility.
• Converts AHP pairwise comparisons into normalized weights for each metric, ensuring consistency through automated checks.
• Combines fairness and utility into a single objective function using the elicited weights, with utility maximized and fairness metrics minimized.
• Applies Bayesian optimization with Tree of Parzen Estimators (TPE) to search for optimal classification thresholds for privileged and unprivileged groups.
• Scales metrics using domain-specific scales (e.g., 100 for utility, 0.1 for SPD and WAOD) to normalize units and improve optimization stability.
• Imposes legal constraints by rejecting solutions where the disparate impact ratio falls outside [0.8, 1.25] to ensure compliance with U.S. EEOC guidelines.

Experimental results

Research questions

• RQ1 How can policy makers’ preferences on fairness versus utility trade-offs be systematically elicited in high-stakes AI applications?
• RQ2 What is the impact of different preference weightings on the resulting model’s fairness and utility metrics?
• RQ3 Can a joint optimization framework effectively balance multiple fairness criteria (SPD and WAOD) and model utility under user-defined constraints?
• RQ4 How does the framework perform across diverse preference profiles, including extreme cases like maximizing utility or minimizing fairness deviations?
• RQ5 Can the framework be extended to aggregate preferences from multiple stakeholders with conflicting priorities?

Key findings

• The framework successfully identified a model with maximum utility (130) when preferences prioritized utility over fairness, achieving 95.0% utility and -0.06 SPD.
• When minimizing absolute SPD, the model achieved near-perfect fairness with SPD = 0.00 and WAOD = 0.14, demonstrating effective bias mitigation.
• For the balanced preference setting, the model achieved 95.1% utility, SPD = -0.01, and WAOD = 0.14, showing a strong trade-off between fairness and utility.
• The Bayesian optimization process reliably converged to optimal thresholds across all preference settings, as confirmed by consistent metric outcomes in Table 1.
• The framework effectively reconciled conflicting fairness criteria and enabled transparent, principled decision-making across diverse stakeholder perspectives.