[Paper Review] No computation without representation: Avoiding data and algorithm biases through diversity

Caitlin Kuhlman, Latifa Jackson+1 more|arXiv (Cornell University)|Feb 26, 2020

Ethics and Social Impacts of AISocial Sciences61 references17 citations

TL;DR

This paper argues that ethical AI cannot be achieved without diversifying the computing community itself, as lack of representation in data science perpetuates structural biases in datasets and algorithms. By integrating underrepresented voices through targeted education, mentoring, and collaboration—especially with minority-serving institutions—it is possible to embed fairness from the start, reducing algorithmic bias and fostering equitable socio-technical systems.

ABSTRACT

The emergence and growth of research on issues of ethics in AI, and in particular algorithmic fairness, has roots in an essential observation that structural inequalities in society are reflected in the data used to train predictive models and in the design of objective functions. While research aiming to mitigate these issues is inherently interdisciplinary, the design of unbiased algorithms and fair socio-technical systems are key desired outcomes which depend on practitioners from the fields of data science and computing. However, these computing fields broadly also suffer from the same under-representation issues that are found in the datasets we analyze. This disconnect affects the design of both the desired outcomes and metrics by which we measure success. If the ethical AI research community accepts this, we tacitly endorse the status quo and contradict the goals of non-discrimination and equity which work on algorithmic fairness, accountability, and transparency seeks to address. Therefore, we advocate in this work for diversifying computing as a core priority of the field and our efforts to achieve ethical AI practices. We draw connections between the lack of diversity within academic and professional computing fields and the type and breadth of the biases encountered in datasets, machine learning models, problem formulations, and interpretation of results. Examining the current fairness/ethics in AI literature, we highlight cases where this lack of diverse perspectives has been foundational to the inequity in treatment of underrepresented and protected group data. We also look to other professional communities, such as in law and health, where disparities have been reduced both in the educational diversity of trainees and among professional practices. We use these lessons to develop recommendations that provide concrete steps for the computing community to increase diversity.

Motivation & Objective

• To address the root cause of algorithmic bias by confronting the lack of diversity in computing and data science communities.
• To highlight how underrepresentation in AI research leads to blind spots in fairness, accountability, and transparency.
• To demonstrate that diverse perspectives are essential for identifying and mitigating structural inequalities in data and model design.
• To propose actionable strategies—such as mentoring, educational partnerships, and community-based research collaborations—to build a more inclusive AI research ecosystem.
• To advocate for a bottom-up, community-centered approach to diversity that empowers underrepresented groups as leaders in ethical AI.

Proposed method

• Analyzing existing literature in algorithmic fairness to identify cases where lack of diverse perspectives led to biased model design.
• Drawing parallels between underrepresentation in computing and systemic biases in datasets and algorithms.
• Examining successful diversity initiatives in law and healthcare to inform best practices for computing.
• Highlighting the role of mentoring and affinity workshops—such as the BPDM workshop—at Howard University in building community and technical capacity.
• Proposing educational collaborations with minority-serving institutions (MSIs) to increase representation in AI research pipelines.
• Advocating for research partnerships with community domain experts to ensure models reflect real-world social equity concerns.

Experimental results

Research questions

• RQ1 How does the underrepresentation of certain demographic groups in computing research contribute to persistent algorithmic biases?
• RQ2 In what ways do homogeneous research teams fail to detect or address structural inequalities in data and model design?
• RQ3 What role do mentoring and community-building programs play in increasing diversity and fairness in AI research?
• RQ4 How can partnerships with minority-serving institutions and community experts improve the fairness and relevance of AI systems?
• RQ5 What systemic changes in education and research culture are needed to achieve sustainable diversity in AI and ethical computing?

Key findings

• The lack of diversity in computing communities directly contributes to the emergence of biased datasets and algorithms, as homogeneous teams fail to recognize or account for structural inequalities.
• Diverse perspectives are essential for identifying and mitigating indirect and explicit discrimination in algorithmic systems, especially when protected attributes are not directly used.
• Mentoring and affinity workshops—such as the BPDM workshop at Howard University—successfully foster community, build technical skills, and increase representation among underrepresented groups in data science.
• Educational collaborations with minority-serving institutions can help grow a more representative talent pipeline in AI and data science.
• Community-based research partnerships with domain experts from underrepresented communities lead to more equitable and contextually relevant model development.
• Without intentional efforts to diversify the AI research community, efforts to achieve fairness through algorithmic manipulation alone will remain insufficient and short-lived.