[Paper Review] Heuristics for publishing dynamic content as structured data with schema.org

Elias Kärle, Dieter Fensel|arXiv (Cornell University)|Aug 17, 2018

Service-Oriented Architecture and Web Services22 references3 citations

TL;DR

This paper proposes context-aware heuristics for publishing dynamic, fast-changing product and service data as structured schema.org markup, avoiding full materialization of all data instances. By using abstractions, selective instance publication, or web service references, the approach improves scalability and performance while minimizing server load and data disclosure risks—demonstrating that heuristic-based publication outperforms full data exposure in real-world scenarios.

ABSTRACT

Publishing fast changing dynamic data as open data on the web in a scalable manner is not trivial. So far the only approaches describe publishing as much data as possible, which then leads to problems, like server capacity overload, network latency or unwanted knowledge disclosure. With this paper we show ways how to publish dynamic data in a scalable, meaningful manner by applying context-dependent publication heuristics. The outcome shows that the application of the right publication heuristics in the right domain can improve the publication performance significantly. Good knowledge about the domain help choosing the right publication heuristic and hence lead to very good publication results.

Motivation & Objective

• To address the challenge of publishing large volumes of fast-changing dynamic data (e.g., product prices, availabilities) as machine-readable schema.org structured data.
• To overcome scalability issues such as server overload, network latency, and unintended data disclosure from full data materialization.
• To develop context-dependent heuristics that enable meaningful, efficient, and secure publication of dynamic content on the web.
• To evaluate heuristic-based approaches against full data publication, focusing on performance, maintainability, and practical feasibility.
• To guide practitioners in selecting appropriate heuristics based on domain-specific characteristics and data dimensionality.

Proposed method

• Proposing four heuristic strategies: (1) publishing abstract data with references to web services for concrete data lookup; (2) publishing a single representative instance with implied availability of others; (3) selective instance-level materialization based on dimensionality; and (4) using web service descriptions to expose search or retrieval interfaces.
• Using schema.org vocabulary (sdo) to annotate products with properties like name, description, image, and sdo/Offer for price and availability.
• Introducing the concept of 'long dimensions'—dimensions with excessive cardinality that make full materialization infeasible—and proposing heuristics to avoid them.
• Leveraging Semantic Web services to decouple abstract schema.org annotations from concrete data, enabling on-demand retrieval.
• Designing a validation pipeline that checks both schema compliance and adherence to publication heuristics to prevent data leakage and inconsistency.
• Extending the semantify.it platform to support selective, heuristic-driven annotation publishing instead of full-site annotation.

Experimental results

Research questions

• RQ1 How can dynamic, high-velocity data be published as structured schema.org data without overwhelming server resources or exposing sensitive information?
• RQ2 What are the performance and scalability trade-offs of different heuristic strategies for publishing dynamic data?
• RQ3 When and how should abstractions or single instances be used instead of full materialization of all data variations?
• RQ4 How can domain-specific knowledge guide the selection of optimal publication heuristics?
• RQ5 What metrics or criteria define a 'long dimension' in data publication, and how can such dimensions be safely ignored?

Key findings

• No single heuristic strategy outperforms all others universally; the best approach is highly dependent on the specific use case and domain characteristics.
• The evaluation confirmed that heuristic-based approaches consistently outperform full materialization in terms of scalability, performance, and resource efficiency.
• Publishing abstract data with web service references significantly reduces server load while preserving machine readability and enabling on-demand data access.
• Selective instance-level materialization is effective when dimensions are manageable, but becomes complex when determining which dimensions are 'long' or should be excluded.
• Information loss from abstraction is mitigated by providing access to underlying web services, ensuring consumers can retrieve concrete data when needed.
• The integration of publication heuristics into platforms like semantify.it enables fine-grained control over which data subsets are published, improving practical deployment.