[Paper Review] Structured Factored Inference: A Framework for Automated Reasoning in Probabilistic Programming Languages

Avi Pfeffer, Brian E. Ruttenberg+1 more|arXiv (Cornell University)|Jun 10, 2016

Bayesian Modeling and Causal Inference17 references3 citations

TL;DR

This paper introduces Structured Factored Inference (SFI), a framework for automated probabilistic inference in probabilistic programming languages that decomposes complex models into sub-models, applies tailored inference algorithms to each, and combines results via factor-based propagation. SFI achieves performance comparable to exact inference with significantly faster runtime than approximate methods, demonstrating near-optimal accuracy and scalability on real-world models.

ABSTRACT

Reasoning on large and complex real-world models is a computationally difficult task, yet one that is required for effective use of many AI applications. A plethora of inference algorithms have been developed that work well on specific models or only on parts of general models. Consequently, a system that can intelligently apply these inference algorithms to different parts of a model for fast reasoning is highly desirable. We introduce a new framework called structured factored inference (SFI) that provides the foundation for such a system. Using models encoded in a probabilistic programming language, SFI provides a sound means to decompose a model into sub-models, apply an inference algorithm to each sub-model, and combine the resulting information to answer a query. Our results show that SFI is nearly as accurate as exact inference yet retains the benefits of approximate inference methods.

Motivation & Objective

• To address the computational bottleneck of probabilistic inference in large, complex real-world models.
• To reduce the burden of manual algorithm selection and configuration by enabling automated, model-aware inference strategy composition.
• To provide a sound, semantically grounded framework for decomposing probabilistic models into sub-models with manageable interfaces.
• To enable hybrid inference by combining exact and approximate methods on different sub-models, improving both speed and accuracy.
• To lay the foundation for a general, extensible automated inference system in probabilistic programming.

Proposed method

• Leverages probabilistic programming semantics to identify decomposition points in a model, enabling structured sub-model extraction.
• Applies inference algorithms independently to each sub-model, reducing them to joint distributions over query-relevant variables.
• Uses factors to represent and propagate results between sub-models, enabling consistent combination of inference outcomes.
• Employs heuristic strategies to guide algorithm selection per sub-model based on structural complexity (e.g., tree-width, factor size).
• Supports hybrid inference by combining exact methods (e.g., variable elimination) and approximate methods (e.g., Gibbs sampling, belief propagation) on different sub-models.
• Uses a hierarchical decomposition strategy to create sub-models with minimal interface size, improving scalability.

Experimental results

Research questions

• RQ1 Can a framework automatically decompose complex probabilistic models into sub-models to enable efficient inference?
• RQ2 Can combining exact and approximate inference algorithms on different sub-models achieve both high accuracy and fast performance?
• RQ3 How does the choice of decomposition granularity and inference strategy affect inference efficiency and accuracy?
• RQ4 Can heuristic-based algorithm selection in SFI outperform monolithic inference approaches on real-world models?
• RQ5 To what extent can SFI reduce the need for manual algorithm tuning in probabilistic modeling?

Key findings

• The hybrid SFI strategy (VE/BP–15) achieved faster runtime than flat BP–30 and flat VE, while maintaining negligible accuracy difference.
• On the seismic monitoring model, hierarchical VE outperformed the hybrid strategy in runtime, indicating that hand-crafted elimination orders can surpass heuristics.
• The SFI framework completed all tests on the seismic model, whereas full VE failed due to memory limits, demonstrating robustness.
• The hybrid method (VE/BP–30) was more accurate than flat BP–30 and faster than flat VE, showing superiority in both accuracy and speed.
• Even with simple heuristics, SFI achieved performance nearly as accurate as exact inference and faster than standard approximate methods.
• The framework enables scalable inference on models where exact methods fail due to memory or time constraints.