[Paper Review] BigCQ: A large-scale synthetic dataset of competency question patterns formalized into SPARQL-OWL query templates

Dawid Wiśniewski, Jędrzej Potoniec+1 more|arXiv (Cornell University)|May 20, 2021

Topic Modeling21 references4 citations

TL;DR

This paper introduces BigCQ, a large-scale synthetic dataset of 77,575 competency question (CQ) templates paired with 549 SPARQL-OWL query templates, automatically generated from frequent OWL axiom shapes. The dataset enables automated CQ-to-query translation and Glossary-of-Terms extraction, with 63.89% coverage of real-world CQs and 45.74% coverage of SPARQL-OWL queries from existing corpora, published with reusable scripts for extensibility.

ABSTRACT

Competency Questions (CQs) are used in many ontology engineering methodologies to collect requirements and track the completeness and correctness of an ontology being constructed. Although they are frequently suggested by ontology engineering methodologies, the publicly available datasets of CQs and their formalizations in ontology query languages are very scarce. Since first efforts to automate processes utilizing CQs are being made, it is of high importance to provide large and diverse datasets to fuel these solutions. In this paper, we present BigCQ, the biggest dataset of CQ templates with their formalizations into SPARQL-OWL query templates. BigCQ is created automatically from a dataset of frequently used axiom shapes. These pairs of CQ templates and query templates can be then materialized as actual CQs and SPARQL-OWL queries if filled with resource labels and IRIs from a given ontology. We describe the dataset in detail, provide a description of the process leading to the creation of the dataset and analyze how well the dataset covers real-world examples. We also publish the dataset as well as scripts transforming axiom shapes into pairs of CQ patterns and SPARQL-OWL templates, to make engineers able to adapt the process to their particular needs.

Motivation & Objective

• To address the scarcity of publicly available, large-scale datasets linking natural language competency questions (CQs) to formal SPARQL-OWL queries in ontology engineering.
• To enable automation of CQ-based tasks such as Glossary-of-Terms extraction and CQ-to-query translation using machine learning.
• To generate a scalable, extensible dataset of CQ and SPARQL-OWL template pairs from frequently occurring OWL axiom shapes.
• To improve coverage and quality of CQ-to-query mappings for use in training and evaluating NLP and knowledge representation systems.

Proposed method

• Automatically generating CQ templates from a set of frequently occurring OWL axiom shapes using rule-based verbalization patterns.
• Mapping each CQ template to a corresponding SPARQL-OWL query template using formalized syntactic and semantic rules.
• Using a database of synonyms and lexical variations to enrich CQ templates and increase linguistic diversity.
• Materializing full CQ-SPARQL-OWL pairs by substituting ontology-specific labels and IRIs into the templates.
• Validating coverage by comparing against real-world datasets: CQ2SPARQL-OWL and CORAL.
• Publishing the dataset and transformation scripts on GitHub and via Zenodo for reproducibility and extensibility.

Experimental results

Research questions

• RQ1 How can a large-scale, synthetic dataset of CQ templates and their SPARQL-OWL query counterparts be systematically generated from common OWL axiom patterns?
• RQ2 To what extent does the generated dataset cover real-world competency questions and SPARQL-OWL queries from existing corpora?
• RQ3 What are the limitations of the current template generation process in handling complex or rare query patterns?
• RQ4 Can the dataset serve as effective training data for automated CQ-to-query translation and Glossary-of-Terms extraction systems?

Key findings

• BigCQ contains 77,575 distinct CQ templates and 549 distinct SPARQL-OWL query templates, with an average of 171.68 CQ templates per query template.
• The dataset achieves 63.89% coverage of CQs from the CORAL corpus and 45.74% coverage of SPARQL-OWL queries from the CQ2SPARQL-OWL dataset.
• The most frequently occurring constructs in the query templates are owl:Restriction (504/549), owl:onProperty (504/549), and owl:intersectionOf (319/549).
• Two major uncovered query patterns are nested property chains and queries involving owl:disjointWith, which together account for ~25% of unsupported cases.
• The dataset is extensible: the authors provide open-source scripts to adapt the pipeline to new axiom shapes, verbalization rules, and synonym databases.
• The dataset is published with a persistent DOI and is available on GitHub for reuse, extension, and integration into NLP and ontology engineering pipelines.