[Paper Review] Data Preparation in Agriculture Through Automated Semantic Annotation -- Basis for a Wide Range of Smart Services

Julian Klose, Markus Schröder+3 more|arXiv (Cornell University)|Nov 15, 2019

Semantic Web and Ontologies1 references4 citations

TL;DR

This paper proposes Wikinormia, a collaborative, semantic annotation platform that enables automated data preparation in agriculture by standardizing heterogeneous machine-generated data formats through RDF-based definitions. By transforming raw data into structured spatial, time-series, and semantic graph representations via parsers, the SDSD platform empowers smart services to access standardized, semantically enriched data while preserving farmer data sovereignty, enabling scalable, interoperable digital agriculture workflows.

ABSTRACT

Modern agricultural technology and the increasing digitalisation of such processes provide a wide range of data. However, their efficient and beneficial use suffers from legitimate concerns about data sovereignty and control, format inconsistencies and different interpretations. As a proposed solution, we present Wikinormia, a collaborative platform in which interested participants can describe and discuss their own new data formats. Once a finalized vocabulary has been created, specific parsers can semantically process the raw data into three basic representations: spatial information, time series and semantic facts (agricultural knowledge graph). Thanks to publicly accessible definitions and descriptions, developers can easily gain an overview of the concepts that are relevant to them. A variety of services will then (subject to individual access rights) be able to query their data simply via a query interface and retrieve results. We have already implemented this proposed solution in a prototype in the SDSD (Smart Data - Smart Services) project and demonstrate the benefits with a range of representative services. This provides an efficient system for the cooperative, flexible digitalisation of agricultural workflows.

Motivation & Objective

• Address the challenge of agricultural data fragmentation due to proprietary, machine-specific data formats from diverse manufacturers.
• Enable cross-manufacturer data interoperability and semantic understanding without relying on slow standardization processes.
• Empower farmers with data sovereignty by allowing fine-grained access control over their data shared with third-party services.
• Facilitate the development of smart agricultural services by abstracting away low-level data format complexities through a unified query interface.
• Create a scalable, extensible platform for collaborative data format description and semantic enrichment in agriculture using Semantic Web technologies.

Proposed method

• Utilize Wikinormia, a wiki-like platform, to collaboratively define agricultural data formats using the Resource Description Framework (RDF) as machine-readable, human-readable semantic models.
• Implement parsers that interpret raw data (e.g., JSON, XML, binary) based on Wikinormia-defined schemas to extract and structure information.
• Store processed data in a polyglot persistence architecture: spatial data in MongoDB with spatial indexing, time series in Cassandra for scalability, and semantic facts in a Stardog triplestore.
• Model agricultural entities (fields, machines, tasks) as RDF resources with unique URIs, enabling semantic linking and deduplication via spatial equality (SameAs relationships).
• Enable smart services to query structured data via a standardized interface, bypassing direct access to raw, heterogeneous data formats.
• Integrate with external systems like agrirouter for real-time data ingestion and leverage OpenStreetMap for field boundary inference when needed.

Experimental results

Research questions

• RQ1 How can agricultural data from heterogeneous, manufacturer-specific sources be semantically standardized and made interoperable in a timely manner?
• RQ2 To what extent can a collaborative, community-driven platform for data format description outperform traditional, slow standardization processes in dynamic agricultural contexts?
• RQ3 How can data sovereignty and fine-grained access control be preserved while enabling third-party smart services to consume processed agricultural data?
• RQ4 What is the feasibility of automatically separating relevant field operation data from continuous machine recordings (e.g., road trips) using semantic and spatial reasoning?
• RQ5 Can a unified, extensible data platform reduce the complexity and cost of developing smart agricultural services for small and medium providers?

Key findings

• The Wikinormia platform successfully enables collaborative, extensible, and machine-processable description of agricultural data formats, including support for ISOXML and region-specific formats like NRW agricultural application data.
• Parsers built on Wikinormia definitions can automatically transform raw data into three standardized data models: spatial, time-series, and semantic graphs.
• The polyglot storage architecture effectively manages diverse data types—spatial data via MongoDB, time series via Cassandra, and semantic facts via Stardog triplestore—enabling efficient querying and integration.
• The system enables semantic deduplication of field instances using spatial overlap and SameAs relationships, improving data consistency across sources.
• A prototype time series separation service successfully isolates field-specific operations from non-field data (e.g., travel paths) using field boundaries from OpenStreetMap or existing data, demonstrating practical utility.
• The SDSD platform enables smart services to retrieve required data via a uniform interface without handling raw formats, significantly reducing integration complexity and costs for service developers.