[Paper Review] OpenExtract: Automated Data Extraction for Systematic Reviews in Health

Jim Achterberg, Bram van Dijk+9 more|arXiv (Cornell University)|Mar 6, 2026

Biomedical Text Mining and Ontologies0 citations

TL;DR

OpenExtract is an open-source pipeline that uses large language models to automatically extract data for systematic reviews in health, achieving precision and recall above 0.8 compared with human researchers.

ABSTRACT

This study presents OpenExtract, an open-source pipeline for automated data extraction in large-scale systematic literature reviews. The pipeline queries large language models (LLMs) to predict data entries based on relevant sections of scientific articles. To test the efficacy of OpenExtract, we apply it to a systematic literature review in digital health and compare its outputs with those of human researchers. OpenExtract achieves precision and recall scores of > 0.8 in this task, indicating that it can be effective at extracting data automatically and efficiently. OpenExtract: https://github.com/JimAchterbergLUMC/OpenExtract.

Motivation & Objective

• Motivate the need for scalable data extraction in health systematic reviews.
• Propose an open-source pipeline to automate data extraction using LLMs.
• Evaluate OpenExtract against human researchers on a digital health systematic review.

Proposed method

• OpenExtract queries large language models to predict data entries from relevant article sections.
• The pipeline automates data extraction tasks typically performed by researchers.
• Comparative evaluation against human researchers using a digital health systematic review.

Experimental results

Research questions

• RQ1 Can an automated pipeline accurately extract predefined data entries for health systematic reviews using LLMs?
• RQ2 How does OpenExtract compare to human researchers in terms of precision and recall on data extraction tasks in health literature?
• RQ3 Is the open-source approach effective for large-scale systematic reviews?

Key findings

• OpenExtract achieves precision and recall scores of > 0.8 when extracting data entries for a digital health systematic review.
• The pipeline demonstrates efficiency in producing data extraction outputs comparable to human researchers.
• The study validates OpenExtract as a viable automated tool for large-scale systematic reviews in health.