[Paper Review] Building Representative Corpora from Illiterate Communities: A Review of Challenges and Mitigation Strategies for Developing Countries

Stephanie Hirmer, Alycia Leonard+2 more|arXiv (Cornell University)|Feb 4, 2021

ICT in Developing Communities87 references4 citations

TL;DR

This paper proposes a framework for building representative NLP corpora from illiterate, rural communities in low-income countries by identifying ethical and methodological challenges in data collection and offering practical mitigation strategies. It emphasizes voice-based data collection, community engagement, and ethical stewardship to reduce bias and ensure inclusivity in NLP research.

ABSTRACT

Most well-established data collection methods currently adopted in NLP depend on the assumption of speaker literacy. Consequently, the collected corpora largely fail to represent swathes of the global population, which tend to be some of the most vulnerable and marginalised people in society, and often live in rural developing areas. Such underrepresented groups are thus not only ignored when making modeling and system design decisions, but also prevented from benefiting from development outcomes achieved through data-driven NLP. This paper aims to address the under-representation of illiterate communities in NLP corpora: we identify potential biases and ethical issues that might arise when collecting data from rural communities with high illiteracy rates in Low-Income Countries, and propose a set of practical mitigation strategies to help future work.

Motivation & Objective

• Address the underrepresentation of illiterate, rural populations in NLP corpora due to reliance on literacy and internet access in standard data collection methods.
• Identify ethical and methodological challenges in collecting data from illiterate communities in low-income countries (LICs), particularly in sub-Saharan Africa.
• Propose practical, context-sensitive mitigation strategies to reduce bias and ensure ethical data collection and stewardship.
• Promote the inclusion of marginalized, non-literate speakers in NLP research to improve model fairness and global representativeness.
• Support sustainable development outcomes by ensuring data-driven NLP systems benefit the most vulnerable populations.

Proposed method

• Adopt voice-based data collection methods, such as audio interviews and oral storytelling, to bypass literacy requirements.
• Use local gatekeepers and community representatives to build trust and ensure cultural appropriateness in recruitment and consent processes.
• Implement verbal consent procedures with clear, localized explanations to ensure informed participation in illiterate communities.
• Apply anonymization techniques to protect identities, locations, and culturally sensitive information in collected data.
• Implement data safeguarding protocols, including secure storage and in-place handling, with transparent communication to participants.
• Use non-financial incentives or avoid remuneration altogether to prevent coercion and align with local norms, reducing power imbalances.

Experimental results

Research questions

• RQ1 What ethical and methodological challenges arise when collecting NLP corpora from illiterate, rural communities in low-income countries?
• RQ2 How do standard NLP data collection methods—such as crowdsourcing, social media scraping, and written surveys—fail to represent illiterate populations?
• RQ3 What strategies can mitigate bias and power imbalances in data collection from marginalized, non-literate communities?
• RQ4 How can researchers ensure data privacy, informed consent, and participant autonomy in low-literacy, rural settings?
• RQ5 What mechanisms can reduce research fatigue and ensure community feedback loops in repeated data collection efforts?

Key findings

• Standard NLP data collection methods—like crowdsourcing and web scraping—exclude illiterate populations due to implicit assumptions of literacy and internet access.
• High illiteracy and limited infrastructure in rural low-income countries lead to significant demographic misrepresentation in existing corpora.
• Verbal consent, local gatekeepers, and community engagement are essential for ethical and effective data collection in illiterate settings.
• Anonymization of names, locations, and cultural identifiers is critical to protect participant privacy, especially in regions with weak democratic institutions.
• Failure to share research findings with communities can lead to research fatigue and reduced future participation, undermining data quality.
• Non-financial incentives or no remuneration may be more ethical and context-appropriate than monetary payments in communities where gift-giving can imply coercion.