[Paper Review] Blang: Bayesian declarative modelling of general data structures and inference via algorithms based on distribution continua

Alexandre Bouchard‐Côté, Kevin Chern+6 more|arXiv (Cornell University)|Dec 22, 2019

Bayesian Methods and Mixture Models8 references4 citations

TL;DR

Blang is a Bayesian probabilistic programming language that enables declarative modeling of arbitrary, non-standard data structures—such as combinatorial objects, phylogenies, and alignments—using a BUGS-like syntax. It automates inference via scalable Monte Carlo methods, including non-reversible MCMC and sequential change of measure, by leveraging distribution continua and dynamic sampler composition, achieving high performance on complex state spaces without manual algorithm tuning.

ABSTRACT

Consider a Bayesian inference problem where a variable of interest does not take values in a Euclidean space. These "non-standard" data structures are in reality fairly common. They are frequently used in problems involving latent discrete factor models, networks, and domain specific problems such as sequence alignments and reconstructions, pedigrees, and phylogenies. In principle, Bayesian inference should be particularly well-suited in such scenarios, as the Bayesian paradigm provides a principled way to obtain confidence assessment for random variables of any type. However, much of the recent work on making Bayesian analysis more accessible and computationally efficient has focused on inference in Euclidean spaces. In this paper, we introduce Blang, a domain specific language and library aimed at bridging this gap. Blang allows users to perform Bayesian analysis on arbitrary data types while using a declarative syntax similar to BUGS. Blang is augmented with intuitive language additions to create data types of the user's choosing. To perform inference at scale on such arbitrary state spaces, Blang leverages recent advances in sequential Monte Carlo and non-reversible Markov chain Monte Carlo methods.

Motivation & Objective

• To address the gap in Bayesian inference tools for non-Euclidean, combinatorial data structures such as phylogenies, alignments, and networks.
• To simplify the development and deployment of Bayesian models on arbitrary state spaces by abstracting away low-level sampling complexity.
• To enable scalable, high-performance inference on complex models by automating the integration of advanced Monte Carlo algorithms.
• To support reproducible, modular, and composable model development through a full-featured software stack with IDE support and dependency management.
• To democratize access to high-performance inference by embedding sophisticated sampling schemes (e.g., non-reversible MCMC, SMC) directly into a high-level modeling language.

Proposed method

• Blang uses a domain-specific language built on Xtext to provide a declarative, BUGS-like syntax for specifying probabilistic models over arbitrary data types.
• It introduces a type system with annotations such as @Samplers and @ConnectedFactor to enable dynamic, reflection-based discovery and composition of MCMC samplers at runtime.
• The framework automatically constructs interpolating families of probability distributions (distribution continua) between prior and posterior, enabling efficient non-reversible MCMC and sequential change of measure (SCM) algorithms.
• It leverages adaptive, parallelized inference schemes including parallel tempering and non-reversible MCMC, which are automatically applied based on model structure and data type.
• The system supports automatic inference pipeline construction via a modular sampler registry, where samplers are selected and instantiated based on the types of latent variables and their connected factors.
• Model development is enhanced through full IDE support, static typing, just-in-time compilation, and integration with standard software engineering practices including testing, profiling, and version control.

Experimental results

Research questions

• RQ1 How can Bayesian inference be efficiently and scalably performed on arbitrary, non-standard data structures such as combinatorial objects and discrete graphs?
• RQ2 Can a declarative modeling language automate the selection and composition of high-performance MCMC and SMC samplers without requiring manual algorithm tuning?
• RQ3 To what extent can distribution continua be used to unify and accelerate inference across diverse state spaces in probabilistic modeling?
• RQ4 How can a modern software stack with full IDE support and modular dependency management improve the usability and reproducibility of Bayesian modeling in practice?
• RQ5 Can advanced Monte Carlo methods like non-reversible MCMC and sequential change of measure be abstracted into a high-level language without sacrificing performance or expressivity?

Key findings

• Blang successfully enables Bayesian inference on complex, non-standard data structures such as simplices, integers, and combinatorial objects using a declarative, high-level syntax.
• The system automatically constructs and composes samplers based on model annotations and factor graph structure, eliminating the need for manual sampler implementation for each model type.
• The use of distribution continua enables efficient non-reversible MCMC and sequential change of measure (SCM) inference, which scale well on parallel architectures.
• The framework achieves high performance on benchmark models, including those involving phylogenetic trees and sequence alignments, by leveraging adaptive, specialized samplers without user intervention.
• The integration of full software engineering tooling—such as static typing, debugging, profiling, and dependency management—significantly improves model maintainability and reproducibility.
• The system demonstrates that advanced Monte Carlo inference can be abstracted into a high-level language while preserving computational efficiency, as evidenced by the automatic construction of 2 samplers (RealSliceSampler and IntSliceSampler) in the example pipeline.