[Paper Review] Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation

Cody Blakeney, Nathaniel Huish+2 more|arXiv (Cornell University)|Jun 15, 2021

Adversarial Robustness in Machine Learning26 references4 citations

TL;DR

This paper introduces two novel metrics, Combined Error Variance (CEV) and Symmetric Distance Error (SDE), to quantitatively evaluate bias in pruned neural networks. It demonstrates that knowledge distillation effectively mitigates pruning-induced bias—even with unbalanced datasets—and reveals that model similarity, measured via SVCCA, strongly correlates with reduced bias, offering an explanation for why bias emerges during pruning.

ABSTRACT

In recent years the ubiquitous deployment of AI has posed great concerns in regards to algorithmic bias, discrimination, and fairness. Compared to traditional forms of bias or discrimination caused by humans, algorithmic bias generated by AI is more abstract and unintuitive therefore more difficult to explain and mitigate. A clear gap exists in the current literature on evaluating and mitigating bias in pruned neural networks. In this work, we strive to tackle the challenging issues of evaluating, mitigating, and explaining induced bias in pruned neural networks. Our paper makes three contributions. First, we propose two simple yet effective metrics, Combined Error Variance (CEV) and Symmetric Distance Error (SDE), to quantitatively evaluate the induced bias prevention quality of pruned models. Second, we demonstrate that knowledge distillation can mitigate induced bias in pruned neural networks, even with unbalanced datasets. Third, we reveal that model similarity has strong correlations with pruning induced bias, which provides a powerful method to explain why bias occurs in pruned neural networks. Our code is available at https://github.com/codestar12/pruning-distilation-bias

Motivation & Objective

• To address the lack of effective metrics for evaluating bias in pruned neural networks.
• To investigate whether knowledge distillation can mitigate pruning-induced bias, especially in unbalanced datasets.
• To explain the root causes of bias in pruned models by analyzing model similarity using SVCCA.
• To provide a quantitative, objective, and scalable alternative to existing metrics like PIEs.

Proposed method

• Propose Combined Error Variance (CEV) as a metric that quantifies error variance across classes to assess bias prevention quality.
• Introduce Symmetric Distance Error (SDE) as a symmetric, normalized error metric that enables direct comparison between two models.
• Apply state-of-the-art knowledge distillation techniques to pruned models to reduce bias, even when training data is class-imbalanced.
• Use Singular Vector Canonical Correlation Analysis (SVCCA) to measure similarity between feature representations of pruned and original models across layers.
• Compare CEV and SDE scores across multiple pruning methods and sparsity levels to evaluate bias mitigation performance.
• Use SVCCA distances as a predictor of CEV to explain the relationship between model similarity and induced bias.

Experimental results

Research questions

• RQ1 How can we objectively and quantitatively evaluate bias in pruned neural networks beyond existing metrics like PIEs?
• RQ2 Can knowledge distillation effectively reduce pruning-induced bias, particularly in datasets with class imbalance?
• RQ3 What is the relationship between model similarity and the level of bias in pruned neural networks?
• RQ4 Does high top-1 accuracy correlate with low bias in pruned models, or are other metrics more predictive?
• RQ5 Can SVCCA-based similarity measures explain why certain pruned models induce more bias than others?

Key findings

• CEV and SDE outperform PIEs by enabling direct, objective comparison between two models without requiring large populations of pruned models.
• Knowledge distillation significantly reduces pruning-induced bias in pruned models, even when trained on unbalanced datasets, as confirmed by lower CEV and SDE scores.
• Model similarity, measured via SVCCA, is the strongest predictor of CEV across all sparsity levels and pruning methods, indicating that higher similarity to the original model leads to less induced bias.
• Top-1 accuracy is a poor indicator of bias: models with the highest top-1 accuracy on biased datasets often rank among the worst in terms of CEV and SDE scores.
• The correlation between SVCCA distance and CEV holds across different datasets (CIFAR100, CIFAR10, Imagenette, Imagewoof) and architectures (ResNet, VGG16), confirming its generalizability.
• Pruned models with high feature similarity to the original model consistently exhibit lower bias, suggesting that preserving representation fidelity is key to fairness.