[Paper Review] Characterising Bias in Compressed Models

Sara Hooker, Nyalleng Moorosi+3 more|arXiv (Cornell University)|Oct 6, 2020

Advanced Neural Network Applications39 references46 citations

TL;DR

This paper shows that model compression via pruning and quantization leaves overall accuracy largely intact but amplifies bias by increasing errors on a small subset of difficult examples, termed Compression Identified Exemplars (CIE), and proposes CIE as a human-in-the-loop auditing tool.

ABSTRACT

The popularity and widespread use of pruning and quantization is driven by the severe resource constraints of deploying deep neural networks to environments with strict latency, memory and energy requirements. These techniques achieve high levels of compression with negligible impact on top-line metrics (top-1 and top-5 accuracy). However, overall accuracy hides disproportionately high errors on a small subset of examples; we call this subset Compression Identified Exemplars (CIE). We further establish that for CIE examples, compression amplifies existing algorithmic bias. Pruning disproportionately impacts performance on underrepresented features, which often coincides with considerations of fairness. Given that CIE is a relatively small subset but a great contributor of error in the model, we propose its use as a human-in-the-loop auditing tool to surface a tractable subset of the dataset for further inspection or annotation by a domain expert. We provide qualitative and quantitative support that CIE surfaces the most challenging examples in the data distribution for human-in-the-loop auditing.

Motivation & Objective

• Assess how pruning and quantization affect predictive bias and fairness across protected attribute subgroups.
• Quantify error disparities (FPR/FNR) between compressed and non-compressed models on gender and age attributes.
• Introduce and validate the concept of Compression Identified Exemplars (CIE) as a label-agnostic auditing tool.
• Explore surrogate measures (Modal CIE and Taxicab CIE) to surface challenging examples without protected attribute labels.
• Propose human-in-the-loop auditing workflow leveraging CIE to inspect high-bias regions of the data.

Proposed method

• Train ResNet-18 on CelebA for binary blond vs non-blond classification.
• Apply pruning to end-sparsity levels t in {0.3,0.5,0.7,0.9,0.95,0.99} during training and fix masks at inference.
• Apply post-training 8-bit quantization (hybrid dynamic range and fixed-point) to compare against baseline.
• Evaluate sub-group performance using unitary (Male, Young) and intersectional (Young Male) attributes with metrics including top-1 accuracy, FPR, and FNR.
• Define Compression Identified Exemplars (CIE) as examples where compressed and non-compressed label predictions diverge (Modal CIE and Taxicab CIE).
• Repeat experiments across 30 models per compression level to ensure statistical significance.

Experimental results

Research questions

• RQ1 Does model compression amplify algorithmic bias on underrepresented attribute subgroups?
• RQ2 How do false positive and false negative rates change under pruning and quantization across subgroups?
• RQ3 Can CIE (Modal and Taxicab) surface challenging instances without attribute labels for auditing?
• RQ4 Are CIEs indicative of the hardest examples for both compressed and non-compressed models?
• RQ5 Can a human-in-the-loop workflow using CIE effectively audit compressed models in limited-annotation regimes?

Key findings

• Overall top-1 accuracy remains high under compression (e.g., 94.73% baseline, 94.07% at 90% pruning, 93.39% at 95% pruning).
• Compression consistently amplifies disparate treatment of underrepresented subgroups across pruning levels.
• At 95% pruning, female false positive rate increases by 49.54% relative to baseline, while male false positives increase by 6.32% relative to baseline.
• Higher compression levels worsen performance on underrepresented attributes, particularly on minority intersectional groups.
• CIE measures (Modal and Taxicab) identify a small, hard subset of data where accuracy drops sharply under compression (e.g., 49.82%/55.35% accuracy on Modal/Taxicab CIEs at certain thresholds).
• CIEs over-index on underrepresented attributes and surface challenging examples for human auditing without requiring protected-attribute labels.