[Paper Review] An Introduction to Probabilistic Programming

Staton, Sam|arXiv (Cornell University)|Sep 27, 2018

Bayesian Modeling and Causal Inference108 references110 citations

TL;DR

This paper provides a comprehensive graduate-level introduction to probabilistic programming, presenting a principled framework for building and reasoning about probabilistic models using programming languages. It covers foundational concepts in model-based reasoning, inference algorithms (e.g., Gibbs sampling, SMC, Hamiltonian Monte Carlo), and advanced techniques like differentiable and deep probabilistic programming, with a focus on composability, inference modularity, and integration with machine learning.

ABSTRACT

This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understanding of either or, ideally, both probabilistic machine learning and programming languages. We start with a discussion of model-based reasoning and explain why conditioning is a foundational computation central to the fields of probabilistic machine learning and artificial intelligence. We then introduce a first-order probabilistic programming language (PPL) whose programs correspond to graphical models with a known, finite, set of random variables. In the context of this PPL we introduce fundamental inference algorithms and describe how they can be implemented. We then turn to higher-order probabilistic programming languages. Programs in such languages can define models with dynamic computation graphs, which may not instantiate the same set of random variables in each execution. Inference requires methods that generate samples by repeatedly evaluating the program. Foundational algorithms for this kind of language are discussed in the context of an interface between program executions and an inference controller. Finally we consider the intersection of probabilistic and differentiable programming. We begin with a discussion of automatic differentiation, and how it can be used to implement efficient inference methods based on Hamiltonian Monte Carlo. We then discuss gradient-based maximum likelihood estimation in programs that are parameterized using neural networks, how to amortize inference using by learning neural approximations to the program posterior, and how language features impact the design of deep probabilistic programming systems.

Motivation & Objective

• To provide a foundational understanding of probabilistic programming for researchers with background in machine learning and programming languages.
• To explain how probabilistic programs can be compiled into graphical models and factor graphs for efficient inference.
• To introduce inference algorithms—like likelihood weighting, Metropolis-Hastings, and SMC—that operate via repeated program evaluation.
• To explore higher-order probabilistic programming with dynamic computation graphs and the challenges it poses for inference.
• To integrate differentiable programming with probabilistic programming, enabling gradient-based inference and deep generative modeling.

Proposed method

• Designs a first-order probabilistic programming language (FOPPL) with static computation graphs and finite random variables, mapping programs to graphical models.
• Introduces a messaging interface between model and inference controller to decouple model specification from inference strategy.
• Employs continuation-passing style (CPS) transformations to enable modular and composable inference in higher-order PPLs.
• Applies automatic differentiation to enable Hamiltonian Monte Carlo (HMC) inference in differentiable probabilistic programs.
• Uses stochastic-gradient methods and amortized inference to scale learning in deep probabilistic models.
• Integrates neural networks as proposals in inference, enabling end-to-end training of generative models with probabilistic programs.

Experimental results

Research questions

• RQ1 How can probabilistic programming be formalized as a first-order language with static computation graphs and finite random variables?
• RQ2 What inference algorithms are suitable for higher-order probabilistic programs with dynamic computation graphs?
• RQ3 How can inference be modularized through a messaging interface between models and inference controllers?
• RQ4 What role does automatic differentiation play in enabling efficient gradient-based inference in differentiable probabilistic programs?
• RQ5 How can deep neural networks be used to amortize inference and scale learning in probabilistic programs?

Key findings

• The paper establishes a principled framework for compiling probabilistic programs into graphical models and factor graphs, enabling systematic application of standard inference algorithms.
• It demonstrates that inference in higher-order PPLs requires repeated program evaluation and can be modularized via a messaging interface, enabling separation of concerns.
• The integration of automatic differentiation with probabilistic programming enables efficient HMC inference, significantly improving mixing and convergence in high-dimensional spaces.
• Amortized inference using neural networks reduces the computational cost of posterior approximation, enabling scalable learning in complex models.
• The paper shows that deep probabilistic programming systems can be designed to support both generative modeling and inference through end-to-end differentiable and modular components.
• Empirical results from case studies (e.g., in fMRI analysis, topic modeling) show that the proposed methods achieve state-of-the-art performance in posterior approximation and model learning.