[Paper Review] SAT-Based PAC Learning of Description Logic Concepts

Balder ten Cate, Maurice Funk+2 more|arXiv (Cornell University)|May 15, 2023

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces bounded fitting, a SAT-based PAC learning framework for learning description logic concepts in ${\cal E\!\!>LH}^{r}$ ontologies. By iteratively searching for size-bounded concepts using a SAT solver, the method provides formal PAC generalization guarantees and enables efficient learning via SPELL, a system outperforming existing learners like ELTL in runtime and scalability on larger concepts.

ABSTRACT

We propose bounded fitting as a scheme for learning description logic concepts in the presence of ontologies. A main advantage is that the resulting learning algorithms come with theoretical guarantees regarding their generalization to unseen examples in the sense of PAC learning. We prove that, in contrast, several other natural learning algorithms fail to provide such guarantees. As a further contribution, we present the system SPELL which efficiently implements bounded fitting for the description logic $\mathcal{ELH}^r$ based on a SAT solver, and compare its performance to a state-of-the-art learner.

Motivation & Objective

• To address the lack of formal generalization guarantees in existing DL concept learning systems.
• To develop a learning framework that ensures PAC learning properties through size-bounded concept search.
• To implement an efficient, SAT-based system for learning ${\cal E\!\!>L}$ concepts under ${\cal E\!\!>LH}^{r}$ ontologies.
• To empirically evaluate the generalization and performance of bounded fitting against state-of-the-art learners.
• To analyze the sample efficiency of alternative learning approaches, including refinement operators and most-specific/least-specific fitting algorithms.

Proposed method

• Bounded fitting incrementally searches for a concept of increasing size bound that fits all training examples under an ontology.
• The size-restricted fitting problem is reduced to SAT, leveraging efficient SAT solvers for scalable computation.
• The approach is grounded in Occam's razor, ensuring that the returned concept is small, which guarantees PAC generalization.
• A system called SPELL is implemented to perform bounded fitting using a SAT solver, with ontology and concept constraints encoded in CNF.
• The method uses a k-universal database approximation per round to simulate universal database behavior for each size bound k.
• Theoretical analysis compares the sample efficiency of bounded fitting against other algorithms, including refinement-based and most-specific fitting methods.

Experimental results

Research questions

• RQ1 Can bounded fitting provide formal PAC learning guarantees for description logic concept learning?
• RQ2 How does the sample efficiency of bounded fitting compare to other learning algorithms like refinement operators or most-specific fitting?
• RQ3 What is the performance of a SAT-based implementation (SPELL) relative to existing DL learners like ELTL?
• RQ4 Can bounded fitting learn larger concepts more efficiently than existing systems?
• RQ5 Do both bounded fitting and refinement-based learners generalize well to unseen data, even with small training sets?

Key findings

• Bounded fitting provides formal PAC generalization guarantees because it is an Occam algorithm, ensuring sample complexity depends linearly on 1/δ, 1/ε, and the size of the target concept.
• The size-restricted fitting problem for ${\cal E\!\!>L}$ concepts under ${\cal E\!\!">LH}^{r}$ ontologies is NP-complete, enabling efficient SAT-based encoding.
• SPELL outperforms ELTL in runtime across all datasets, with significantly lower running times, especially for larger concepts.
• The running time of SPELL is primarily influenced by the size of the target concept, allowing it to scale to larger queries than ELTL.
• Both SPELL and ELTL generalize well to unseen data even on very small training samples, though the reason for ELTL's success remains unclear due to heuristic design.
• Most-specific fitting and standard refinement operators (without artificial Occam constraints) are not sample-efficient, failing to provide PAC guarantees.