[Paper Review] Bayesian Workflow

Andrew Gelman, Aki Vehtari+8 more|arXiv (Cornell University)|Nov 3, 2020

Bayesian Modeling and Causal Inference137 citations

TL;DR

This paper lays out a comprehensive Bayesian workflow—from model construction and fitting to checking, comparison, and software practices—illustrated by golf putting and planetary motion examples.

ABSTRACT

The Bayesian approach to data analysis provides a powerful way to handle uncertainty in all observations, model parameters, and model structure using probability theory. Probabilistic programming languages make it easier to specify and fit Bayesian models, but this still leaves us with many options regarding constructing, evaluating, and using these models, along with many remaining challenges in computation. Using Bayesian inference to solve real-world problems requires not only statistical skills, subject matter knowledge, and programming, but also awareness of the decisions made in the process of data analysis. All of these aspects can be understood as part of a tangled workflow of applied Bayesian statistics. Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model comparison. We review all these aspects of workflow in the context of several examples, keeping in mind that in practice we will be fitting many models for any given problem, even if only a subset of them will ultimately be relevant for our conclusions.

Motivation & Objective

• Define Bayesian workflow as an integration of model building, inference, and model checking/improvement.
• Advocate pre-data design, prior predictive checks, and generative modeling to understand and constrain analyses.
• Promote iterative model building, debugging, and comparison of multiple models to understand uncertainty.
• Highlight practical computational concerns, diagnostics, and strategies for working with probabilistic programming (Stan).
• Provide guidelines and examples to systematize Bayesian analyses in real-world problems.

Proposed method

• Describe initial model construction using templates and modular components for flexibility.
• Emphasize scaling and transforming parameters to facilitate interpretation and hierarchical modeling.
• Advocate prior predictive checking to assess prior implications before observing data.
• Discuss generative versus non-generative modeling and implications for predictive checks.
• Outline fitting with modern algorithms (primarily Hamiltonian Monte Carlo) and diagnostic practices (e.g., Rhat, R*).
• Present examples of workflow through Golf putting and Planetary motion to illustrate iterative model development.

Experimental results

Research questions

• RQ1 What comprises a practical and systematic Bayesian workflow beyond mere inference?
• RQ2 How can prior choices, model expansions, and computational diagnostics be integrated to produce trustworthy inferences?
• RQ3 What are effective strategies for building, checking, and comparing multiple Bayesian models in real-world problems?
• RQ4 How should one manage computation, model expansion, and data incorporation within an iterative workflow?

Key findings

• Bayesian workflow encompasses model building, inference, checking, and iterative improvement, not just posterior computation.
• Prior predictive checks and generative modeling help anticipate data behavior and guide priors and model structure.
• Modular model construction and parameter scaling facilitate interpretability and hierarchical modeling.
• Diagnostics for inference (e.g., warmup, mixing, convergence) are essential for trustworthy results and efficient exploration of models.
• Comparison and visualization of multiple models are central to understanding uncertainty and robustness of conclusions.
• Examples (Golf putting; Planetary motion) demonstrate how models evolve with new data and computational challenges, underscoring iterative workflow.