[Paper Review] Entrofy Your Cohort: A Data Science Approach to Candidate Selection

Daniela Huppenkothen, Brian McFee+1 more|arXiv (Cornell University)|May 8, 2019

Names, Identity, and Discrimination Research36 references4 citations

TL;DR

This paper introduces Entrofy, a data science algorithm that automates diversity-driven cohort selection after a blinded merit review. By optimizing for predefined demographic and categorical criteria, Entrofy ensures transparent, auditable, and bias-resistant selection of high-merit candidates, as validated in simulations and a case study of Astro Hack Week 2016.

ABSTRACT

Selecting a cohort from a set of candidates is a common task within and beyond academia. Admitting students, awarding grants, choosing speakers for a conference are situations where human biases may affect the make-up of the final cohort. We propose a new algorithm, Entrofy, designed to be part of a larger decision making strategy aimed at making cohort selection as just, quantitative, transparent, and accountable as possible. We suggest this algorithm be embedded in a two-step selection procedure. First, all application materials are stripped of markers of identity that could induce conscious or sub-conscious bias. During blind review, the committee selects all applicants, submissions, or other entities that meet their merit-based criteria. This often yields a cohort larger than the admissible number. In the second stage, the target cohort can be chosen from this meritorious pool via a new algorithm and software tool. Entrofy optimizes differences across an assignable set of categories selected by the human committee. Criteria could include gender, academic discipline, experience with certain technologies, or other quantifiable characteristics. The Entrofy algorithm yields the computational maximization of diversity by solving the tie-breaking problem with provable performance guarantees. We show how Entrofy selects cohorts according to pre-determined characteristics in simulated sets of applications and demonstrate its use in a case study. This cohort selection process allows human judgment to prevail when assessing merit, but assigns the assessment of diversity to a computational process less likely to be beset by human bias. Importantly, the stage at which diversity assessments occur is fully transparent and auditable with Entrofy. Splitting merit and diversity considerations into their own assessment stages makes it easier to explain why a given candidate was selected or rejected.

Motivation & Objective

• Address human biases in academic and professional cohort selection processes, especially in admissions, grants, and conference speaker selection.
• Separate merit assessment from diversity considerations to reduce implicit bias and improve fairness.
• Develop a computational tool that enables transparent, auditable, and reproducible selection of diverse cohorts from a pool of meritorious candidates.
• Provide a scalable, quantitative alternative to intuitive or heuristic-based selection methods commonly used in committee decisions.
• Demonstrate the feasibility and effectiveness of algorithmic diversity maximization in real-world academic settings, such as workshops and conferences.

Proposed method

• Implement a two-stage selection process: first, a blinded merit review to identify all qualified candidates, removing names, affiliations, and identity markers.
• Second, apply the Entrofy algorithm to select the final cohort from the merit-qualified pool based on user-defined diversity criteria.
• Use an optimization framework that maximizes diversity across multiple categorical attributes (e.g., gender, country, discipline, technical experience) with provable performance guarantees.
• Formulate the selection as a constrained optimization problem that balances diversity targets with the requirement to maintain high merit.
• Leverage combinatorial optimization techniques to solve the tie-breaking problem in a way that is both efficient and mathematically sound.
• Make the entire process transparent and auditable by logging all decisions and criteria used in the Entrofy algorithm.

Experimental results

Research questions

• RQ1 Can an algorithmic approach improve fairness and transparency in cohort selection compared to traditional committee-based methods?
• RQ2 To what extent can diversity be systematically maximized in a cohort while preserving merit-based selection?
• RQ3 How does the two-stage process—blinded merit review followed by algorithmic diversity selection—affect the composition and accountability of the final cohort?
• RQ4 Can the Entrofy algorithm reliably produce optimal or near-optimal diversity outcomes in real-world scenarios with multiple categorical constraints?
• RQ5 What are the practical implications of using Entrofy in academic workshops, conferences, and admissions processes with diverse candidate pools?

Key findings

• Entrofy successfully produced cohorts that matched predefined diversity targets in both simulated datasets and real-world applications, such as Astro Hack Week 2016.
• The algorithm achieved optimal or near-optimal diversity scores in all tested scenarios, demonstrating strong performance guarantees under the optimization framework.
• The two-stage process—blinded merit review followed by algorithmic diversity selection—resulted in more transparent and auditable outcomes than traditional committee-based selection.
• The method reduced the risk of implicit bias in final selections by decoupling merit evaluation from diversity assessment.
• The case study of Astro Hack Week 2016 showed that Entrofy could produce a balanced cohort across gender, career stage, and geographic origin while maintaining high merit standards.
• The open-source software and fully reproducible code (available on GitHub) enable replication and adoption across academic and professional settings.