[Paper Review] Coverage and metadata completeness and accuracy of African research publications in OpenAlex: A comparative analysis

Patricia Alonso-Álvarez, Nees Jan van Eck|arXiv (Cornell University)|Sep 2, 2024

Publishing and Scholarly Communication4 citations

TL;DR

This study evaluates OpenAlex's coverage and metadata quality for African research publications by comparing it with Scopus, Web of Science (WoS), and African Journals Online (AJOL). OpenAlex demonstrates superior coverage of African publications and strong metadata completeness for publication and author fields, though affiliation, reference, and funding data are less complete; accuracy improves significantly for publications also indexed in Scopus and WoS, indicating strong potential as a sustainable alternative to proprietary databases for African research analytics.

ABSTRACT

Unlike traditional proprietary data sources such as Scopus and the Web of Science (WoS), OpenAlex emphasizes its comprehensiveness. This study analyzes OpenAlex coverage and metadata completeness and accuracy of African research publications. To achieve this, OpenAlex is compared with Scopus, WoS, and African Journals Online (AJOL). First, we examine the coverage of African research publications in OpenAlex relative to Scopus, WoS, and AJOL. Then, we assess and compare the availability and accuracy of metadata in OpenAlex, Scopus, and WoS. The findings indicate that OpenAlex offers the most extensive publication coverage. In terms of metadata, OpenAlex provides high coverage for publication and author information, though its coverage of affiliations, references, and funder information is comparatively lower. Metadata accuracy is similarly high for publication and author fields, while affiliation, reference, and funding information show higher rates of missing or incomplete data. Notably, the results demonstrate that both metadata availability and accuracy in OpenAlex improve significantly for publications also indexed in Scopus and WoS. These findings suggest that OpenAlex has the potential to replace proprietary data sources for certain types of analyses. However, for some metadata fields, there remains a trade-off between extensiveness and accuracy.

Motivation & Objective

• To evaluate OpenAlex's coverage of African research publications relative to established databases like Scopus, WoS, and AJOL.
• To assess the completeness and accuracy of metadata in OpenAlex, particularly for publication, author, affiliation, reference, and funding fields.
• To investigate whether publications indexed in Scopus and WoS show improved metadata quality in OpenAlex.
• To determine the feasibility of using OpenAlex as a sustainable, comprehensive alternative to proprietary research databases for African scholarly output.

Proposed method

• Conducted a comparative analysis of African research publications across OpenAlex, Scopus, WoS, and AJOL using a shared dataset of publications.
• Measured coverage by comparing the number of African publications identified in each database.
• Assessed metadata completeness by evaluating the proportion of records with available data in each metadata field (e.g., title, authors, affiliations, references, funders).
• Evaluated metadata accuracy by cross-referencing records in OpenAlex with those in Scopus and WoS, treating them as reference standards.
• Focused on publications also indexed in Scopus and WoS to examine whether dual indexing improves metadata quality in OpenAlex.
• Used statistical comparison methods to quantify differences in coverage and metadata quality across databases.

Experimental results

Research questions

• RQ1 How does OpenAlex's coverage of African research publications compare to Scopus, WoS, and AJOL?
• RQ2 To what extent is metadata completeness and accuracy in OpenAlex comparable to that of Scopus and WoS for key fields such as author, affiliation, references, and funding?
• RQ3 Does the inclusion of a publication in Scopus or WoS correlate with higher metadata completeness and accuracy in OpenAlex?
• RQ4 Can OpenAlex serve as a viable, comprehensive alternative to proprietary databases like Scopus and WoS for African research analytics?

Key findings

• OpenAlex exhibits the most extensive coverage of African research publications compared to Scopus, WoS, and AJOL.
• Metadata completeness in OpenAlex is high for publication and author fields, but significantly lower for affiliations, references, and funding information.
• Metadata accuracy is high for publication and author fields in OpenAlex, but lower for affiliations, references, and funders due to missing or incomplete data.
• Publications indexed in both OpenAlex and Scopus or WoS show substantially improved metadata completeness and accuracy in OpenAlex, indicating a positive impact of dual indexing.
• Despite gaps in some metadata fields, OpenAlex demonstrates strong potential to replace proprietary databases for large-scale analyses of African scholarly output.
• The trade-off between comprehensiveness and accuracy is evident, particularly for affiliation and funding metadata, which remain incomplete despite OpenAlex’s broad coverage.