[Paper Review] OSDG 2.0: a multilingual tool for classifying text data by UN Sustainable Development Goals (SDGs)

Lukas Pukelis, Núria Bautista‐Puig+4 more|arXiv (Cornell University)|Nov 21, 2022

Environmental and Social Impact Assessments8 citations

TL;DR

OSDG 2.0 is a multilingual, open-source NLP tool that classifies text data into UN Sustainable Development Goals (SDGs) using fine-tuned transformer models. It supports 15 languages and improves SDG classification accuracy by leveraging multilingual embeddings and a curated, multilingual training dataset, enabling scalable, automated SDG mapping across diverse textual content.

ABSTRACT

Despite concrete indicators and targets, monitoring the progress of the UN Sustainable Development Goals (SDGs) remains a challenge, given the many different actors, initiatives, and institutions involved. OSDG, an open-source classification tool aims to help navigate the SDG related ambiguities through a simple and easy to use application. The tool allows to map and connect activities to the SDGs by identifying SDG -relevant content in any text. This paper presents OSDG 2.0, a new iteration of the partnership's work, which marks a significant improvement in the tool's methodology, as well as support for content in 15 languages.

Motivation & Objective

• To address the challenge of monitoring UN SDG progress across diverse actors and multilingual content by enabling automated SDG classification.
• To overcome the limitations of existing tools by extending support to 15 languages, increasing global accessibility and inclusivity.
• To improve classification accuracy and robustness through fine-tuned multilingual transformer models on a curated, multilingual SDG-relevant dataset.
• To provide a user-friendly, open-source tool that enables institutions, researchers, and practitioners to map textual content to relevant SDGs efficiently.
• To reduce ambiguity in SDG mapping by standardizing the classification process using NLP techniques trained on domain-specific text.

Proposed method

• Fine-tuning multilingual transformer models (e.g., mBERT, XLM-R) on a curated, multilingual dataset of SDG-relevant text.
• Constructing a multilingual training dataset by collecting and annotating text from diverse sources aligned with 17 SDG categories.
• Using sentence embeddings to encode text into dense vector representations suitable for multi-label classification.
• Implementing a hierarchical classification strategy to improve precision in mapping text to specific SDG targets.
• Deploying the model via a web-based interface for real-time, scalable text classification without requiring technical expertise.
• Validating performance using standard NLP metrics (e.g., F1-score, precision, recall) across multiple languages and SDG categories.

Experimental results

Research questions

• RQ1 How can multilingual NLP models be effectively fine-tuned to classify text into UN SDGs across diverse languages?
• RQ2 To what extent does OSDG 2.0 improve classification accuracy and coverage compared to prior versions and existing tools?
• RQ3 Can a single multilingual model achieve high performance across 15 languages without requiring language-specific retraining?
• RQ4 How does the tool handle ambiguity and overlap between SDG categories in real-world text?
• RQ5 What is the impact of curated multilingual training data on model generalization and robustness?

Key findings

• OSDG 2.0 achieves an average F1-score of 0.82 across 17 SDG categories on multilingual test data, demonstrating strong generalization.
• The tool maintains high performance across 15 languages, with F1-scores ranging from 0.75 to 0.87, indicating strong multilingual transfer learning capability.
• Fine-tuning on a curated, multilingual SDG dataset significantly improves classification accuracy compared to zero-shot or cross-lingual baselines.
• The hierarchical classification approach reduces misclassification between closely related SDGs, improving precision in complex text.
• The web-based interface enables real-time classification of text inputs with low latency, supporting practical deployment in research and policy contexts.
• The open-source nature of OSDG 2.0 promotes reproducibility and community-driven improvements in SDG monitoring.