[Paper Review] Stochastic And-Or Grammars: A Unified Framework and Logic Perspective

Kewei Tu|arXiv (Cornell University)|Jun 2, 2015

Bayesian Modeling and Causal Inference24 references4 citations

TL;DR

This paper introduces a unified, data-agnostic framework for stochastic And-Or Grammars (AOGs) that models compositional and reconfigurable structures across diverse data types like images, videos, and events. It proposes a domain-independent inference algorithm with tractability under composition sparsity and establishes logical interpretations of AOGs as probabilistic logic and stochastic logic programs, linking them to statistical relational learning.

ABSTRACT

Stochastic And-Or grammars (AOG) extend traditional stochastic grammars of language to model other types of data such as images and events. In this paper we propose a representation framework of stochastic AOGs that is agnostic to the type of the data being modeled and thus unifies various domain-specific AOGs. Many existing grammar formalisms and probabilistic models in natural language processing, computer vision, and machine learning can be seen as special cases of this framework. We also propose a domain-independent inference algorithm of stochastic context-free AOGs and show its tractability under a reasonable assumption. Furthermore, we provide two interpretations of stochastic context-free AOGs as a subset of probabilistic logic, which connects stochastic AOGs to the field of statistical relational learning and clarifies their relation with a few existing statistical relational models.

Motivation & Objective

• To unify disparate domain-specific stochastic AOGs used in computer vision, NLP, and event modeling into a single, data-agnostic formalism.
• To develop a domain-independent inference algorithm for stochastic context-free AOGs that is tractable under a reasonable composition sparsity assumption.
• To establish formal connections between stochastic AOGs and probabilistic logic frameworks, such as tractable Markov logic and stochastic logic programs.
• To clarify the theoretical relationship between AOGs and existing probabilistic models, including sum-product networks and constraint-based grammars.
• To enable general-purpose learning and inference by abstracting AOGs into a formal, interpretable probabilistic model.

Proposed method

• Proposes a 5-tuple formalism for stochastic context-free AOGs: ⟨Σ, N, S, θ, R⟩, where Σ is terminals, N is nonterminals (And/Or nodes), S is the start symbol, θ maps nodes to parameters, and R defines relations between nodes.
• Introduces parameterized patterns and relations to generalize AOGs across data types, enabling instantiation for images, events, and language.
• Develops a domain-independent inference algorithm based on dynamic programming that computes marginal probabilities by summing over all valid parse trees matching the input.
• Establishes a first-order probabilistic logic interpretation by mapping AOGs to a form of tractable Markov logic, where And/Or rules correspond to logical clauses with probabilistic weights.
• Reduces stochastic context-free AOGs to normalized stochastic logic programs (SLPs), enabling use of existing SLP inference and learning algorithms.
• Uses the composition sparsity assumption to prove tractability of the inference algorithm, ensuring polynomial-time complexity under this condition.

Experimental results

Research questions

• RQ1 How can stochastic AOGs be unified across diverse data types such as images, events, and language, without relying on domain-specific definitions?
• RQ2 Under what conditions is inference in stochastic context-free AOGs tractable, and can this be formalized independently of the data domain?
• RQ3 In what way can stochastic AOGs be interpreted as a form of probabilistic logic, and how do they relate to existing statistical relational models?
• RQ4 Can stochastic AOGs be reduced to known formalisms like stochastic logic programs or tractable Markov logic, and what are the implications for inference and learning?
• RQ5 What is the theoretical relationship between stochastic AOGs and other probabilistic models such as sum-product networks and constraint-based grammars?

Key findings

• The proposed unified AOG framework subsumes various domain-specific models in NLP, computer vision, and machine learning as special cases.
• The domain-independent inference algorithm is tractable under the composition sparsity assumption, ensuring polynomial-time complexity for parsing.
• Stochastic context-free AOGs can be interpreted as a subset of first-order probabilistic logic, resembling tractable Markov logic in structure and inference properties.
• The logical interpretation reveals deep connections between AOGs and existing statistical relational models, particularly in how they handle uncertainty and compositionality.
• Stochastic AOGs can be reduced to normalized stochastic logic programs (SLPs), enabling the application of mature SLP inference and learning algorithms to AOGs.
• The framework enables general-purpose learning and inference across data types by abstracting away domain-specific details while preserving probabilistic and structural expressiveness.