[Paper Review] Leveraging Natural Language Processing and Machine Learning for Evidence-Based Food Security Policy Decision-Making in Data-Scarce Making

Karan Kumar Singh, Nikita Gajbhiye|arXiv (Cornell University)|Mar 20, 2026

Food Security and Health in Diverse Populations0 citations

TL;DR

ZeroHungerAI integrates DistilBERT-based NLP with structured socio-economic indicators to predict food security and optimize policy interventions under data scarcity, with fairness constraints.

ABSTRACT

Food security policy formulation in data-scarce regions remains a critical challenge due to limited structured datasets, fragmented textual reports, and demographic bias in decision-making systems. This study proposes ZeroHungerAI, an integrated Natural Language Processing (NLP) and Machine Learning (ML) framework designed for evidence-based food security policy modeling under extreme data scarcity. The system combines structured socio-economic indicators with contextual policy text embeddings using a transfer learning based DistilBERT architecture. Experimental evaluation on a 1200-sample hybrid dataset across 25 districts demonstrates superior predictive performance, achieving 91 percent classification accuracy, 0.89 precision, 0.85 recall, and an F1 score of 0.86 under imbalanced conditions. Comparative analysis shows a 13 percent performance improvement over classical SVM and 17 percent over Logistic Regression models. Precision Recall evaluation confirms robust minority class detection (average precision around 0.88). Fairness aware optimization reduces demographic parity difference to 3 percent, ensuring equitable rural urban policy inference. The results validate that transformer based contextual learning significantly enhances policy intelligence in low resource governance environments, enabling scalable and bias aware hunger prediction systems.

Motivation & Objective

• Develop a transformer-based NLP pipeline to extract structured policy indicators from unstructured multilingual documents.
• Design a low-resource ML decision-support model that prioritizes interventions with sparse, noisy, and partially labeled data.
• Incorporate robustness and fairness mechanisms to mitigate bias across regions and populations.
• Implement decision-focused learning to optimize intervention allocation under resource constraints.
• Evaluate performance under simulated data scarcity with ablations and baselines.

Proposed method

• A hybrid architecture combining DistilBERT contextual embeddings with normalized structured features via feature fusion.
• A fully connected classifier outputs a binary food security status with confidence scores and policy prioritization insights.
• A fairness-adjusted objective adds demographic parity constraints to the prediction loss.
• Constrained optimization to rank and select regions for intervention under budget limits.
• End-to-end pipeline validated with ablation studies and comparison to baseline models.

Figure 1: Proposed Methodological Layout

Experimental results

Research questions

• RQ1 How can transformer-based NLP extract policy-relevant indicators from unstructured, multilingual documents in data-scarce settings?
• RQ2 Can a low-resource ML model effectively prioritize food security interventions using sparse, noisy, and partially labeled data?
• RQ3 What fairness mechanisms can be integrated to reduce rural–urban demographic bias in policy recommendations?
• RQ4 Does incorporating contextual embeddings with structured indicators improve decision quality under budget constraints?
• RQ5 How does the approach scale with increasing data while maintaining accuracy and fairness?

Key findings

• DistilBERT-based model achieves higher accuracy (94%) than Logistic Regression (79%) and SVM (83%).
• Proposed model reaches AUC around 0.95, precision ~0.92, recall ~0.90, and F1 ~0.91 on a 2000-sample dataset.
• Fairness analysis shows demographic parity gap of 3% (Rural 0.84 vs Urban 0.81).
• Transformer-based learning provides robust minority-class detection under imbalance (average precision ~0.92).
• Compared to classical models, the proposed approach yields substantial performance gains while maintaining fairness constraints.

Figure 2: Backend Operation Layout