[Paper Review] Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices

Manish Raghavan, Solon Barocas+2 more|arXiv (Cornell University)|Jun 21, 2019

Ethics and Social Impacts of AI45 references140 citations

TL;DR

The paper analyzes 18 vendors of algorithmic pre-employment assessments, documenting their development/validation practices, bias claims, and legal considerations, to assess how these tools address fairness.

ABSTRACT

There has been rapidly growing interest in the use of algorithms in hiring, especially as a means to address or mitigate bias. Yet, to date, little is known about how these methods are used in practice. How are algorithmic assessments built, validated, and examined for bias? In this work, we document and analyze the claims and practices of companies offering algorithms for employment assessment. In particular, we identify vendors of algorithmic pre-employment assessments (i.e., algorithms to screen candidates), document what they have disclosed about their development and validation procedures, and evaluate their practices, focusing particularly on efforts to detect and mitigate bias. Our analysis considers both technical and legal perspectives. Technically, we consider the various choices vendors make regarding data collection and prediction targets, and explore the risks and trade-offs that these choices pose. We also discuss how algorithmic de-biasing techniques interface with, and create challenges for, antidiscrimination law.

Motivation & Objective

• Examine how vendors develop, validate, and disclose practices for algorithmic pre-employment assessments.
• Document what vendors disclose about data use, targets, validation, and bias mitigation.
• Evaluate the alignment of vendor practices with antidiscrimination law and industry standards.
• Identify gaps and tensions in industry norms related to bias, validation, and legality.

Proposed method

• Systematically identified vendors offering algorithmic pre-employment assessments via Crunchbase, Upturn, and RedThread Research.
• Extracted publicly disclosed information on assessment types, target variables, training data, validation, and fairness claims from vendor websites.
• Classified practices along the machine learning pipeline to analyze data choices, validation procedures, and bias mitigation.
• Compared vendor claims to legal frameworks such as Title VII, the Uniform Guidelines on Employment Selection Procedures, and the 4/5 rule.
• Synthesized technical challenges around data sources, target outcomes, and alternative assessment formats (e.g., games, video).

Experimental results

Research questions

• RQ1 What types of pre-employment assessments do vendors offer?
• RQ2 How do vendors select target variables and training data for these assessments?
• RQ3 What validation procedures do vendors disclose and how are they tailored to clients?
• RQ4 What bias/fairness claims do vendors make and what methods are used to mitigate bias?
• RQ5 How do legal standards influence vendor practices and the assessment of adverse impact?

Key findings

• 18 vendors offering algorithmic pre-employment assessments were identified and analyzed.
• Assessment types include questions, video interviews, and gameplay, with many vendors offering customizable data usage and targets.
• Validation information is often unclear or limited, though some vendors publish validation studies or demographic audits; others provide little detail.
• A majority of vendors reference bias or adverse impact and mention compliance with the 4/5 rule; de-biasing approaches vary in specificity.
• Two broad fairness strategies emerge: naturally unbiased designs aiming for equal score distributions, and active debiasing that downweights/removes features correlated with protected attributes.
• Industry practices show heterogeneity and evolving norms with limited public transparency on model validity and data provenance.