[Paper Review] Causal Analysis of Author Demographics in Academic Peer Review

Uttamasha Oyshi, Gibson Nkhata+1 more|arXiv (Cornell University)|Feb 25, 2026

scientometrics and bibliometrics research0 citations

TL;DR

The paper uses causal inference with inverse propensity weighting to quantify how author race, gender, and country influence paper acceptance rankings, and shows a fairness-aware model (Fair-PaperRec) can mitigate biases while improving ranking quality.

ABSTRACT

Academic meritocracy is jeopardized by systematic imbalances; for example, whereas Black and Hispanic individuals constitute over 30% of the U.S. population, they represent fewer than 10% of tenured academics in science and engineering. Peer review serves as a crucial gatekeeper in this process, however it encounters ongoing issues over biases that may hinder scientific advancement. The issue is now exacerbated by the growing influence of artificial intelligence (AI) in academic assessment. This paper transcends correlation to quantitatively assess the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings. We utilize a causal inference methodology on a dataset of 530 papers, simulating the academic selection process by employing the prestige of the publication venue as a surrogate for review rank. Our research indicates statistically substantial causal disadvantages for authors from minority racial groups (average treatment effects [ATE]: -0.42 points in ranking), female authors (ATE: -0.25), and those associated with institutions in the Global South (ATE: -0.57). The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes, indicating that such measures are essential for establishing a more equitable and credible scientific environment.

Motivation & Objective

• Quantify the independent causal effects of author race, gender, and country on paper acceptance rankings while controlling for paper quality and institutional prestige.
• Use inverse propensity weighting to estimate average treatment effects (ATE) in a 530-paper dataset from HCI conferences.
• Evaluate a fairness-aware intervention (Fair-PaperRec) for bias mitigation and its impact on ranking utility.
• Analyze intersectional and subgroup-specific biases beyond average effects.
• Provide guidance on fairness interventions in both traditional and AI-assisted peer review contexts.

Proposed method

• Formalize the causal problem within the potential outcomes framework.
• Define treatments as author demographics (Race, Gender, Country) and outcome as paper acceptance ranking.
• Estimate propensity scores via logistic regression using confounders like maximum h-index and institutional prestige.
• Apply inverse propensity weighting (IPW) to balance covariates and estimate ATEs from weighted outcomes.
• Evaluate Fair-PaperRec with a fairness loss that enforces demographic parity across race and country attributes, using NDCG as a utility metric.
• Conduct robustness checks including covariate balance (SMD) and bootstrapped confidence intervals.

Experimental results

Research questions

• RQ1 Do author demographics causally influence paper acceptance rankings when controlling for quality and prestige?
• RQ2 How do causal biases manifest across different subgroups and intersectional identities (e.g., race × gender)?
• RQ3 Can a fairness-aware intervention mitigate identified causal biases without sacrificing, and possibly improving, ranking quality?

Key findings

• Authors from minority racial groups experience a statistically significant causal disadvantage in acceptance rankings (ATE ≈ -0.42 to -0.44).
• Female authors face a statistically significant causal disadvantage (ATE ≈ -0.25 to -0.53 depending on estimation).
• Authors from the Global South show a substantial direct causal effect (ATE ≈ -0.57); intersectional analysis highlights minority-male authors as particularly disadvantaged.
• Fair-PaperRec can drive ATEs toward zero as fairness regularization increases, while also achieving higher NDCG (0.9667 vs 0.9628 baseline).
• Fairness-focused weighting shows a trade-off among fairness components but can improve overall utility and substantially reduce demographic rank gaps.