[Paper Review] Representing Meaning with a Combination of Logical and Distributional Models

Islam Beltagy, Stephen Roller+3 more|arXiv (Cornell University)|May 26, 2015

Topic Modeling92 references5 citations

TL;DR

This paper proposes a hybrid semantic framework that integrates logical and distributional semantics using Markov Logic Networks (MLNs) to improve textual entailment recognition. By combining first-order logic for structural reasoning with distributional word representations for graded similarity, the system achieves state-of-the-art performance (85.1% accuracy) on the SICK dataset, while also releasing a new lexical entailment dataset of 10,213 rules.

ABSTRACT

NLP tasks differ in the semantic information they require, and at this time no single se- mantic representation fulfills all requirements. Logic-based representations characterize sentence structure, but do not capture the graded aspect of meaning. Distributional models give graded similarity ratings for words and phrases, but do not capture sentence structure in the same detail as logic-based approaches. So it has been argued that the two are complementary. We adopt a hybrid approach that combines logic-based and distributional semantics through probabilistic logic inference in Markov Logic Networks (MLNs). In this paper, we focus on the three components of a practical system integrating logical and distributional models: 1) Parsing and task representation is the logic-based part where input problems are represented in probabilistic logic. This is quite different from representing them in standard first-order logic. 2) For knowledge base construction we form weighted inference rules. We integrate and compare distributional information with other sources, notably WordNet and an existing paraphrase collection. In particular, we use our system to evaluate distributional lexical entailment approaches. We use a variant of Robinson resolution to determine the necessary inference rules. More sources can easily be added by mapping them to logical rules; our system learns a resource-specific weight that corrects for scaling differences between resources. 3) In discussing probabilistic inference, we show how to solve the inference problems efficiently. To evaluate our approach, we use the task of textual entailment (RTE), which can utilize the strengths of both logic-based and distributional representations. In particular we focus on the SICK dataset, where we achieve state-of-the-art results.

Motivation & Objective

• To address the limitations of purely logical or distributional semantic models, which fail to capture both sentence structure and graded meaning.
• To develop a unified framework that supports automated inference while incorporating lexical similarity and near-synonymy.
• To enable efficient, scalable probabilistic inference over combined logical and distributional knowledge.
• To evaluate the system on textual entailment, a task requiring deep semantic understanding.
• To release a new, large-scale lexical entailment dataset for future research.

Proposed method

• Represent input sentences as first-order logic formulas using a wide-coverage semantic parser (Boxer) for deep syntactic and semantic analysis.
• Construct a knowledge base by integrating distributional similarity (e.g., word embeddings) with external resources like WordNet and paraphrase databases (PPDB).
• Use probabilistic logic—specifically Markov Logic Networks (MLNs)—to combine logical rules with distributional weights, enabling graded inference.
• Train a supervised entailment rule classifier (eclassif) to identify lexical entailment relations from distributional features.
• Apply inference via weighted model counting in MLNs to compute entailment probabilities efficiently.
• Leverage contextualized distributional features to detect context-sensitive entailments (e.g., 'pour' vs. 'put' for liquids).

Experimental results

Research questions

• RQ1 Can a hybrid system combining logical and distributional semantics outperform purely logical or distributional models in textual entailment?
• RQ2 How effectively can distributional similarity subsume large-scale paraphrase databases like PPDB?
• RQ3 To what extent can distributional features detect context-sensitive lexical entailments not captured by static word vectors?
• RQ4 Can probabilistic logic integrate graded lexical similarity with structural reasoning in a scalable and interpretable way?
• RQ5 How can distributional similarity be used to infer hierarchical relations (e.g., hyponymy) for unknown terms?

Key findings

• The system achieved a state-of-the-art accuracy of 85.1% on the SICK textual entailment test set, outperforming prior approaches.
• Distributional features alone, particularly cosine similarity, were able to capture a significant portion of lexical entailment relations, subsuming much of the PPDB paraphrase database.
• The entailment rule classifier (eclassif) played a critical role in achieving high performance, especially in identifying subtle lexical entailments.
• Adding WordNet (wn) to the system slightly reduced accuracy, suggesting potential noise from over-reliance on lexical hierarchy.
• Contextualized distributional representations showed promise in identifying context-specific entailments, such as 'push' and 'knock' in specific scenarios.
• The system demonstrated modularity and scalability, with potential for extension to question answering and generalized quantifiers via probabilistic logic.