[Paper Review] FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classification

Xiaofeng Lin, Seungbae Kim+1 more|arXiv (Cornell University)|Jul 22, 2022

Face recognition and analysis4 citations

TL;DR

FairGRAPE is a fairness-aware model pruning method that preserves group-specific importance in deep neural networks for face attribute classification. By iteratively pruning low-importance weights while maintaining the relative distribution of group importance across demographic subgroups, it reduces performance disparity by up to 90% compared to state-of-the-art pruning methods, especially at high sparsity rates (e.g., 99% pruning).

ABSTRACT

Existing pruning techniques preserve deep neural networks' overall ability to make correct predictions but may also amplify hidden biases during the compression process. We propose a novel pruning method, Fairness-aware GRAdient Pruning mEthod (FairGRAPE), that minimizes the disproportionate impacts of pruning on different sub-groups. Our method calculates the per-group importance of each model weight and selects a subset of weights that maintain the relative between-group total importance in pruning. The proposed method then prunes network edges with small importance values and repeats the procedure by updating importance values. We demonstrate the effectiveness of our method on four different datasets, FairFace, UTKFace, CelebA, and ImageNet, for the tasks of face attribute classification where our method reduces the disparity in performance degradation by up to 90% compared to the state-of-the-art pruning algorithms. Our method is substantially more effective in a setting with a high pruning rate (99%). The code and dataset used in the experiments are available at https://github.com/Bernardo1998/FairGRAPE

Motivation & Objective

• Address the issue that existing pruning methods amplify accuracy disparities across demographic subgroups by disproportionately removing features important to underrepresented groups.
• Develop a pruning framework that maintains both overall model accuracy and fairness by considering per-group importance of model weights.
• Ensure robustness across diverse datasets and high sparsity levels, particularly in real-world mobile and wearable applications where fairness and efficiency are critical.
• Demonstrate that fairness in compressed models cannot be resolved by data rebalancing alone, necessitating algorithmic intervention during pruning.

Proposed method

• Calculate group-wise importance scores for each network weight using the gradient magnitude squared, $ I_{w,k} = (g_w w)^2 $, where $ g_w $ is derived from average loss over mini-batches.
• Maintain the relative distribution of total importance per demographic group (e.g., race, gender) during pruning by selecting edges that best preserve both magnitude and share of group importance.
• Iteratively prune low-importance weights while re-computing importance scores after each pruning step to ensure dynamic fairness preservation.
• Use a subset of 20% of training data for importance calculation, which was empirically found to yield optimal trade-off between speed and accuracy.
• Apply the method across multiple architectures (e.g., ResNet-34, MobileNet-V2) and datasets (FairFace, UTKFace, CelebA, ImageNet) to ensure generalizability.
• Compare with baseline pruning methods (e.g., SNIP, GraSP, Weight Selection) under identical settings to isolate the fairness benefit of FairGRAPE.

Experimental results

Research questions

• RQ1 To what extent do standard pruning methods exacerbate performance disparities across demographic subgroups in face attribute classification?
• RQ2 Can a pruning strategy be designed to maintain the relative distribution of group importance while achieving high sparsity?
• RQ3 How does FairGRAPE perform compared to state-of-the-art pruning methods in terms of fairness and accuracy at high pruning rates (e.g., 99%)?
• RQ4 Does data rebalancing during gradient computation mitigate pruning-induced bias, or is algorithmic fairness intervention necessary?
• RQ5 How does the layer-wise distribution of group importance change after pruning, and does FairGRAPE preserve balanced importance across layers?

Key findings

• FairGRAPE reduces performance degradation disparity across demographic groups by up to 90% compared to state-of-the-art pruning methods on four datasets.
• At a 99% pruning rate, FairGRAPE maintains significantly lower bias (e.g., $ \rho(\Delta) = 0.78 $ on UTKFace) compared to GraSP ($ \rho(\Delta) = 2.31 $) and SNIP ($ \rho(\Delta) = 1.90 $) when using minority subsets.
• The method achieves the highest accuracy and lowest fairness gap across all sparsity levels, with minimal performance drop even at 99% sparsity.
• Layer-wise analysis shows FairGRAPE preserves balanced importance scores across genders and races, while group-agnostic methods like SNIP and Weight Selection disproportionately prune features important to underrepresented groups.
• Using only minority data for gradient computation does not reduce bias and can even increase it, demonstrating that data rebalancing alone is insufficient to address fairness in pruning.
• The optimal training subset size for importance calculation is 20%, balancing accuracy and computational efficiency.