[Paper Review] Holistic Survey of Privacy and Fairness in Machine Learning

Sina Shaham, Arash Hajisafi+7 more|arXiv (Cornell University)|Jul 28, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper presents a comprehensive, holistic survey of privacy and fairness in machine learning across supervised, unsupervised, semi-supervised, and reinforcement learning. It analyzes the interplay between privacy and fairness, reviews existing architectures and algorithms that jointly optimize both objectives, and identifies key challenges—especially in large language models—while minimizing utility loss.

ABSTRACT

Privacy and fairness are two crucial pillars of responsible Artificial Intelligence (AI) and trustworthy Machine Learning (ML). Each objective has been independently studied in the literature with the aim of reducing utility loss in achieving them. Despite the significant interest attracted from both academia and industry, there remains an immediate demand for more in-depth research to unravel how these two objectives can be simultaneously integrated into ML models. As opposed to well-accepted trade-offs, i.e., privacy-utility and fairness-utility, the interrelation between privacy and fairness is not well-understood. While some works suggest a trade-off between the two objective functions, there are others that demonstrate the alignment of these functions in certain scenarios. To fill this research gap, we provide a thorough review of privacy and fairness in ML, including supervised, unsupervised, semi-supervised, and reinforcement learning. After examining and consolidating the literature on both objectives, we present a holistic survey on the impact of privacy on fairness, the impact of fairness on privacy, existing architectures, their interaction in application domains, and algorithms that aim to achieve both objectives while minimizing the utility sacrificed. Finally, we identify research challenges in achieving privacy and fairness concurrently in ML, particularly focusing on large language models.

Motivation & Objective

• To address the critical gap in understanding the interrelation between privacy and fairness in machine learning, which remains underexplored despite their shared role in trustworthy AI.
• To consolidate and analyze existing methods for integrating privacy and fairness in ML, including pre-processing, in-processing, and post-processing techniques across multiple learning paradigms.
• To examine how privacy impacts fairness and vice versa, identifying both trade-offs and alignments in different ML settings.
• To identify open research challenges, particularly in large language models, where privacy and fairness remain pressing but under-investigated issues.
• To provide a unified framework for minimizing utility loss while simultaneously achieving privacy and fairness in ML pipelines.

Proposed method

• Conducts a systematic review of 150+ works on privacy and fairness in ML, categorizing methods into pre-processing, in-processing, and post-processing strategies.
• Classifies in-processing techniques into adversarial regularizers (e.g., adversarial discriminators for sensitive attribute invariance) and fairness regularizers (e.g., regularization based on bias-free graphs or counterfactual fairness).
• Analyzes semi-supervised learning (SSL) methods, including FairGNN, which uses adversarial learning to make node embeddings invariant to sensitive attributes.
• Evaluates post-processing techniques such as AdaFair and Fair Boosting, which reweight samples or re-sample data to ensure statistical parity and reduce group disparity.
• Proposes a framework for counterfactual fairness in graph neural networks by generating and comparing counterfactual node representations with modified sensitive attributes.
• Integrates evaluation metrics like Benefit Ratio to quantify subgroup accuracy improvements in SSL, enabling fairness-aware model assessment.

Figure 1. Computation of discrimination based on different metrics.

Experimental results

Research questions

• RQ1 How do privacy and fairness interact in machine learning, and in what scenarios do they conflict or align?
• RQ2 What are the most effective architectures and algorithms that simultaneously optimize privacy and fairness while minimizing utility loss?
• RQ3 How do existing pre-processing, in-processing, and post-processing techniques perform across different learning paradigms, including semi-supervised and reinforcement learning?
• RQ4 What are the key challenges in achieving privacy and fairness jointly in large language models?
• RQ5 To what extent can fairness metrics like statistical parity and equalized odds be preserved when enforcing privacy constraints in graph-based and tabular ML models?

Key findings

• Adversarial regularizers in GNNs, such as FairGNN, successfully reduce sensitivity of node embeddings to protected attributes by training a discriminator to predict sensitive features and minimizing its accuracy.
• Fairness regularizers based on bias-free graphs can effectively penalize deviations of learned embeddings from a fairness-preserving reference graph, improving statistical parity in node classification.
• Post-processing techniques like AdaFair and Fair Boosting reduce group disparity by dynamically reweighting misclassified subgroups and ensuring fair representation in sampled training sets.
• In semi-supervised learning, subpopulations with low baseline accuracy often suffer performance degradation after pseudo-labeling, highlighting the risk of unfairness amplification.
• The Benefit Ratio metric effectively captures normalized accuracy improvement across subgroups, revealing that high-baseline groups benefit more from SSL, while low-baseline groups may be disadvantaged.
• Counterfactual fairness approaches that generate and compare modified node representations show promise in achieving invariance to sensitive attributes in graph-based models.