[Paper Review] Learning Fair Representations for Kernel Models

Zilong Tan, Samuel Yeom+2 more|arXiv (Cornell University)|Jun 27, 2019

Ethics and Social Impacts of AI55 references9 citations

TL;DR

This paper proposes a model-aware fair representation learning method for kernel-based models by leveraging sufficient dimension reduction (SDR) in the reproducing kernel Hilbert space (RKHS), enabling provable fairness guarantees for multiple criteria such as statistical parity, equality of opportunity, and equalized odds. The approach introduces a fairness-accuracy trade-off parameterized by principal angles between subspaces and achieves state-of-the-art performance via the first fair Gaussian process (FGP) prior.

ABSTRACT

Fair representations are a powerful tool for establishing criteria like statistical parity, proxy non-discrimination, and equality of opportunity in learned models. Existing techniques for learning these representations are typically model-agnostic, as they preprocess the original data such that the output satisfies some fairness criterion, and can be used with arbitrary learning methods. In contrast, we demonstrate the promise of learning a model-aware fair representation, focusing on kernel-based models. We leverage the classical Sufficient Dimension Reduction (SDR) framework to construct representations as subspaces of the reproducing kernel Hilbert space (RKHS), whose member functions are guaranteed to satisfy fairness. Our method supports several fairness criteria, continuous and discrete data, and multiple protected attributes. We further show how to calibrate the accuracy tradeoff by characterizing it in terms of the principal angles between subspaces of the RKHS. Finally, we apply our approach to obtain the first Fair Gaussian Process (FGP) prior for fair Bayesian learning, and show that it is competitive with, and in some cases outperforms, state-of-the-art methods on real data.

Motivation & Objective

• To address the limitations of model-agnostic fair representation learning by incorporating model-specific inductive biases for kernel methods.
• To develop a fair representation learning framework that guarantees multiple fairness criteria—statistical parity, proxy nondiscrimination, equality of opportunity, and equalized odds—within a unified kernel-based framework.
• To characterize the fairness-accuracy trade-off using principal angles between subspaces of the RKHS, enabling controlled trade-offs.
• To construct the first fair Gaussian process (FGP) prior that ensures fairness through its covariance structure, enabling Bayesian fair learning.
• To empirically validate the method on real-world datasets, demonstrating improved fairness and accuracy over state-of-the-art approaches.

Proposed method

• The method uses sufficient dimension reduction (SDR) to project data into a subspace of the RKHS that captures information unrelated to protected attributes, ensuring fairness under mild conditions.
• It constructs a fair subspace as the orthogonal complement of the SDR projection onto protected attributes, guaranteeing independence from these attributes when the conditional mean is linear.
• A predictive subspace is computed via SDR on the response variable to preserve accuracy, and a third model subspace is defined between the fair and predictive subspaces using a principal angle parameter ε.
• The fairness-accuracy trade-off is formally characterized using canonical angles between subspaces, with the operator norm of the difference in projection operators quantifying the trade-off.
• The method derives a closed-form covariance kernel for a fair Gaussian process (FGP) by exploiting the duality between a GP and its RKHS, ensuring all sample paths lie in the model subspace.
• An algorithm is implemented using QR decomposition and eigenvalue decomposition to compute the fair basis, with SDR applied separately per protected attribute and response variable.

Experimental results

Research questions

• RQ1 Can a model-aware fair representation be constructed for kernel models that provably satisfies multiple fairness criteria?
• RQ2 How can the fairness-accuracy trade-off be formally characterized and controlled in kernel-based learning?
• RQ3 Can the proposed framework be extended to construct a fair Gaussian process prior with guaranteed fairness properties?
• RQ4 How does the performance of the resulting fair model compare to state-of-the-art methods in terms of fairness and accuracy on real-world data?
• RQ5 What is the impact of the principal angle parameter ε on the balance between fairness and predictive accuracy?

Key findings

• The proposed method guarantees fairness under multiple criteria, including statistical parity, equality of opportunity, and equalized odds, by ensuring the learned representation is independent of protected attributes in the RKHS.
• The fairness-accuracy trade-off is precisely characterized using the principal angles between the fair and predictive subspaces, with the operator norm of the difference in projection operators quantified as √(1−γ²_min), where γ_min is the smallest canonical correlation.
• The method achieves competitive or superior accuracy compared to state-of-the-art fair learning baselines on real-world datasets, including UCI and COMPAS.
• The first fair Gaussian process (FGP) prior is constructed by deriving a covariance kernel that ensures all sample paths lie in the model subspace, thereby satisfying the fairness constraints.
• Empirical results show that the FGP achieves rigorous fairness properties while maintaining or improving predictive accuracy, outperforming existing methods in several benchmarks.
• The algorithm efficiently handles multiple, continuous, or discrete protected attributes through separate SDR applications and nullspace computation.