[Paper Review] Why is Compiling Lifted Inference into a Low-Level Language so Effective?

Seyed Mehran Kazemi, David Poole|arXiv (Cornell University)|Jun 14, 2016

Bayesian Modeling and Causal Inference21 references3 citations

TL;DR

This paper investigates why compiling lifted inference into low-level programs (LRC2CPP) achieves orders-of-magnitude speedups over compiling into data structures (as in WFOMC). It demonstrates that the primary speedup stems from efficient compilation and optimization of programs, not from the compilation process itself, and confirms that program execution via compilation is vastly faster than interpreting data structures, even when using the same underlying logic.

ABSTRACT

First-order knowledge compilation techniques have proven efficient for lifted inference. They compile a relational probability model into a target circuit on which many inference queries can be answered efficiently. Early methods used data structures as their target circuit. In our KR-2016 paper, we showed that compiling to a low-level program instead of a data structure offers orders of magnitude speedup, resulting in the state-of-the-art lifted inference technique. In this paper, we conduct experiments to address two questions regarding our KR-2016 results: 1- does the speedup come from more efficient compilation or more efficient reasoning with the target circuit?, and 2- why are low-level programs more efficient target circuits than data structures?

Motivation & Objective

• To determine whether the speedup in LRC2CPP over WFOMC arises from faster compilation or faster reasoning.
• To investigate why reasoning with low-level programs is more efficient than reasoning with data structures.
• To validate the hypothesis that compilation and optimization of programs yield greater performance gains than interpreter-based execution of data structures.
• To isolate the contribution of compilation versus optimization in the observed performance improvements.

Proposed method

• The authors compare end-to-end runtimes of LRC2CPP and WFOMC across three Markov Logic Network benchmarks with varying population sizes.
• They decompose the total time into compilation and reasoning phases to isolate the source of performance gains.
• They generate C++ programs from LRC2CPP and evaluate reasoning performance via three execution modes: compiled with -O3 optimization, compiled without optimization, and interpreted using Ch 7.5.3.
• They compare these execution modes against WFOMC’s data structure-based reasoning, using the same logical models and queries.
• They measure speedup percentages between interpretation and compilation, and between compilation with and without optimization.
• They use statistical analysis to quantify the relative contributions of compilation and optimization to the overall speedup.

Experimental results

Research questions

• RQ1 Does the performance advantage of LRC2CPP over WFOMC stem primarily from faster compilation or faster reasoning with the target circuit?
• RQ2 Why are low-level programs more efficient than data structures as target circuits for lifted inference?
• RQ3 Is the speedup due to the ability to compile and optimize programs, or is it due to inherent properties of the program representation?
• RQ4 To what extent does program compilation contribute to performance gains compared to code optimization?
• RQ5 Can the performance gap between interpreted data structures and compiled programs be quantitatively isolated and measured?

Key findings

• The compilation phase takes nearly the same time in both LRC2CPP and WFOMC, indicating that the speedup is not due to faster compilation.
• For population sizes of 5000, reasoning with LRC2CPP’s programs achieves a 163x speedup over WFOMC’s data structures.
• Reasoning with interpreted C++ programs from LRC2CPP produces runtimes nearly identical to those of WFOMC’s data structures, confirming that the interpreter is the performance bottleneck.
• Compilation of LRC2CPP programs yields an average 175x speedup over interpretation, with 99.7% of the total speedup attributed to compilation rather than optimization.
• Optimization of compiled programs provides only a 2.3x speedup on average compared to non-optimized compilation, indicating minimal impact relative to compilation itself.
• The time complexity of reasoning is identical across both approaches, suggesting that the performance difference is due to constant factors, not asymptotic complexity.