[Paper Review] Direct computation of diagnoses for ontology debugging

Kostyantyn M. Shchekotykhin, Philipp Fleiß+2 more|arXiv (Cornell University)|Sep 5, 2012

Semantic Web and Ontologies11 references3 citations

TL;DR

This paper proposes a direct diagnosis computation approach for ontology debugging that bypasses conflict set generation, enabling efficient diagnosis in ontologies with large numbers of faults—common in ontology matching or learning. By using inverse hitting set and QuickXPlain algorithms, the method achieves comparable or better performance than standard techniques, especially in scenarios where traditional model-based diagnosis fails due to excessive conflict sets.

ABSTRACT

Modern ontology debugging methods allow efficient identification and localization of faulty axioms defined by a user while developing an ontology. The ontology development process in this case is characterized by rather frequent and regular calls to a reasoner resulting in an early user awareness of modeling errors. In such a scenario an ontology usually includes only a small number of conflict sets, i.e. sets of axioms preserving the faults. This property allows efficient use of standard model-based diagnosis techniques based on the application of hitting set algorithms to a number of given conflict sets. However, in many use cases such as ontology alignment the ontologies might include many more conflict sets than in usual ontology development settings, thus making precomputation of conflict sets and consequently ontology diagnosis infeasible. In this paper we suggest a debugging approach based on a direct computation of diagnoses that omits calculation of conflict sets. Embedded in an ontology debugger, the proposed algorithm is able to identify diagnoses for an ontology which includes a large number of faults and for which application of standard diagnosis methods fails. The evaluation results show that the approach is practicable and is able to identify a fault in adequate time.

Motivation & Objective

• To address the scalability limitations of standard model-based diagnosis in ontology debugging when dealing with large numbers of conflict sets.
• To enable efficient debugging of ontologies generated by ontology matching or learning systems, which often produce many faults.
• To develop a direct diagnosis computation method that avoids the computationally expensive precomputation of minimal conflict sets.
• To support interactive debugging by integrating with diagnosis discrimination techniques for oracle-based refinement.
• To demonstrate the feasibility and performance of the approach on large, real-world ontologies with thousands of axioms.

Proposed method

• Introduces Inv-HS-Tree, a breadth-first search algorithm that directly computes diagnoses by inverting the standard model-based diagnosis pipeline.
• Employs Inv-QuickXPlain, a direct diagnosis algorithm that avoids conflict set computation by working backward from diagnosis candidates.
• Uses a hitting set-based approach to identify minimal diagnoses without first computing minimal conflict sets.
• Applies query selection strategies (entropy and split-in-half) to guide interactive diagnosis refinement using an oracle.
• Maintains prior fault probabilities based on confidence scores from ontology matching systems to prioritize likely diagnoses.
• Executes diagnosis computation iteratively for up to n leading diagnoses, enabling efficient exploration of the diagnosis space.

Experimental results

Research questions

• RQ1 Can direct diagnosis computation eliminate the need for precomputing minimal conflict sets in ontology debugging?
• RQ2 How does direct diagnosis performance compare to standard model-based diagnosis in ontologies with high fault density?
• RQ3 Can the direct approach support interactive debugging with oracle queries for diagnosis discrimination?
• RQ4 What is the scalability of the direct diagnosis method on large, real-world ontologies with thousands of axioms?
• RQ5 Does the use of confidence scores from matching systems improve diagnosis selection efficiency?

Key findings

• The direct diagnosis method computed one minimal diagnosis in an average of 25 seconds on benchmark data, comparable to existing greedy systems.
• On unsatisfiable ontologies from the Anatomy problem, the method identified the target diagnosis in an average of 40 seconds.
• Inv-HS-Tree outperformed the standard HS-Tree in several cases, particularly with the entropy-based query selection strategy.
• The entropy measure for query selection outperformed split-in-half by leveraging confidence values from the ontology matcher.
• The method successfully diagnosed ontologies with up to 11,545 axioms, demonstrating scalability in large-scale settings.
• The approach maintained comparable or better performance than standard techniques in cases where conflict set precomputation is infeasible.