[Paper Review] Rethinking the production and publication of machine-reusable expressions of research findings

Markus Stocker, Lauren E. Snyder+7 more|arXiv (Cornell University)|May 21, 2024

Scientific Computing and Data ManagementDecision Sciences3 citations

TL;DR

This paper introduces 'reborn,' a pre-publication framework that embeds machine-reusable scientific knowledge directly into data analysis workflows using the Open Research Knowledge Graph (ORKG). By structuring research findings as FAIR, semantically annotated data during knowledge production—rather than extracting them post-publication—the approach achieves higher accuracy, richer knowledge representation, and simpler technical integration compared to traditional manual or semi-automated post-publication methods.

ABSTRACT

Literature is the primary expression of scientific knowledge and an important source of research data. However, scientific knowledge expressed in narrative text documents is not inherently machine reusable. To facilitate knowledge reuse, e.g. for synthesis research, scientific knowledge must be extracted from articles and organized into databases post-publication. The high time costs and inaccuracies associated with completing these activities manually has driven the development of techniques that automate knowledge extraction. Tackling the problem with a different mindset, we propose a pre-publication approach, known as reborn, that ensures scientific knowledge is born reusable, i.e. produced in a machine-reusable format during knowledge production. We implement the approach using the Open Research Knowledge Graph infrastructure for FAIR scientific knowledge organization. We test the approach with three use cases, and discuss the role of publishers and editors in scaling the approach. Our results suggest that the proposed approach is superior compared to classical manual and semi-automated post-publication extraction techniques in terms of knowledge richness and accuracy as well as technological simplicity.

Motivation & Objective

• To address the limitations of post-publication knowledge extraction, which is time-consuming, error-prone, and often incomplete.
• To enable researchers to produce machine-reusable scientific knowledge at the point of creation, during data analysis, rather than after manuscript publication.
• To improve the FAIRness of scientific knowledge by embedding structured, semantically rich data directly into research workflows using the ORKG infrastructure.
• To reduce reliance on manual or semi-automated post-publication extraction by integrating knowledge structuring into the primary research process.
• To explore the roles of publishers and editors in enabling scalable adoption of pre-publication machine-reusable knowledge production.

Proposed method

• Integrates machine-reusable knowledge production into statistical computing environments (e.g., Python, R) using ORKG’s Python and R libraries for seamless data frame integration.
• Uses LATEX-based authoring with ORKG templates to annotate and structure research findings (e.g., datasets, metrics, scores) as semantically rich, machine-readable data during manuscript preparation.
• Deploys supplementary data (e.g., code, figures, tables) as structured JSON-LD files, linked to the article via DOI metadata using the 'IsSupplementTo' and 'HasPart' relations.
• Employs the TIB Leibniz Data Manager as a centralized, persistent repository for supplementary data, ensuring long-term accessibility and interlinking.
• Enables harvesting of structured data via ORKG’s REST and SPARQL APIs using article DOIs or directory paths, supporting both production and sandbox environments.
• Supports bidirectional interlinking between articles and data using Crossref metadata standards, with optional 'is-supplemented-by' relations for enhanced machine discoverability.

Figure 1: Scientific knowledge expressed in articles is produced as machine-reusable data in computing environments during the data analysis phase of the research lifecycle. Machine-reusable scientific knowledge is deposited in a data repository as supplementary data of the article and interlinked w

Experimental results

Research questions

• RQ1 Can pre-publication structuring of scientific knowledge in machine-readable formats improve the accuracy and richness of knowledge extraction compared to post-publication methods?
• RQ2 How can machine-reusable scientific knowledge be embedded directly into data analysis workflows without disrupting existing research practices?
• RQ3 What role can publishers and editors play in scaling the adoption of pre-publication knowledge structuring across scholarly communication?
• RQ4 To what extent does pre-publication knowledge structuring reduce the time and effort required for synthesis research such as systematic reviews?
• RQ5 How can persistent, interoperable, and FAIR-qualified data be reliably published and discovered using existing DOI and metadata infrastructure?

Key findings

• The reborn approach produces knowledge with higher accuracy and greater structural richness than post-publication extraction, as demonstrated across three real-world use cases.
• Machine-reusable data produced during analysis are natively compatible with ORKG’s Python and R libraries, enabling direct ingestion into data frames for downstream analysis.
• DOI-based interlinking between articles and supplementary JSON-LD data enables reliable, programmatic discovery of harvestable research data using standard APIs such as DataCite’s REST interface.
• Supplementary data deposited in the TIB Leibniz Data Manager are published under CC0 1.0 Universal license, ensuring open, persistent, and reusable access.
• The approach supports both production and sandbox deployment of ORKG contributions, though template compatibility between environments requires decoupling of templates from ORKG’s internal identifier system.
• The ORKG and its components are released under the MIT license, with source code and data openly available on GitLab and via multiple public APIs.

Figure 2: Display of the research finding published by Gentsch et al. in their Figure 1 as a research contribution in ORKG. The overlay expands on the interlinked R script snippet used to implement the respective data analysis. For an interactive experience, we refer readers to the version published