[Paper Review] Going NUTS with ADVI: Exploring various Bayesian Inference techniques with Facebook Prophet

Jovan Krajevski, Biljana Tojtovska|arXiv (Cornell University)|Jan 27, 2026

Forecasting Techniques and Applications0 citations

TL;DR

The paper reimplements Facebook Prophet in PyMC to compare full MCMC, MAP, and Variational Inference techniques on a univariate time-series forecasting task, finding NUTS often converges most reliably while VI offers speed but can misestimate uncertainty.

ABSTRACT

Since its introduction, Facebook Prophet has attracted positive attention from both classical statisticians and the Bayesian statistics community. The model provides two built-in inference methods: maximum a posteriori estimation using the L-BFGS-B algorithm, and Markov Chain Monte Carlo (MCMC) sampling via the No-U-Turn Sampler (NUTS). While exploring various time-series forecasting problems using Bayesian inference with Prophet, we encountered limitations stemming from the inability to apply alternative inference techniques beyond those provided by default. Additionally, the fluent API design of Facebook Prophet proved insufficiently flexible for implementing our custom modeling ideas. To address these shortcomings, we developed a complete reimplementation of the Prophet model in PyMC, which enables us to extend the base model and evaluate and compare multiple Bayesian inference methods. In this paper, we present our PyMC-based implementation and analyze in detail the implementation of different Bayesian inference techniques. We consider full MCMC techniques, MAP estimation and Variational inference techniques on a time-series forecasting problem. We discuss in details the sampling approach, convergence diagnostics, forecasting metrics as well as their computational efficiency and detect possible issues which will be addressed in our future work.

Motivation & Objective

• Motivate flexible Bayesian experimentation beyond Prophet's built-in inference methods.
• Reimplement Facebook Prophet in PyMC to enable modular components and alternative inference techniques.
• Systematically compare full MCMC (MH, NUTS, DMZ), MAP, and Variational Inference (ADVI, FR-ADVI) on a time-series forecasting task.
• Assess convergence diagnostics, forecasting metrics, and computational efficiency across methods.

Proposed method

• Reimplementation of Facebook Prophet in PyMC with a modular API for trend, seasonality, and holidays components.
• Evaluation of three Bayesian inference classes: full MCMC (MH, NUTS, DMZ), MAP (L-BFGS-B), and Variational Inference (ADVI and FR-ADVI).
• Analysis uses a univariate time series (Wikipedia Peyton Manning page views) with default Prophet settings (piecewise linear trend, 25 changepoints, yearly and weekly seasonality).
• Convergence and diagnostic metrics: R-hat, ESS, autocorrelation for MCMC; negative ELBO convergence for VI; forecasting metrics (MSE, RMSE, MAE, MAPE).
• Comparison of sampling and convergence performance across methods, including computational time and effective sample size.

Figure 1: Autocorrelation plot from Metropolis-Hastings for the slope parameter $k$ in the Linear Trend component

Experimental results

Research questions

• RQ1 How do full MCMC, MAP, and VI inference methods compare in accuracy and uncertainty representation for Prophet-like models?
• RQ2 Which Bayesian inference technique provides the best trade-off between forecasting quality and computational efficiency in this setup?
• RQ3 What are the convergence diagnostics and practical limitations of each method when applied to Prophet’s decomposable time-series model?
• RQ4 Does variational inference (ADVI/FR-ADVI) reliably approximate the posterior for Prophet components, and how does it affect uncertainty quantification?

Key findings

• NUTS converges across all parameters with R-hat around 1.0004–1.0010 and high ESS (e.g., 3050.72 for k, 3267.21 for m).
• MH and DMZ fail to converge (R-hat > 1.01 for multiple parameters; ESS < 400 for most parameters), even with large sample sizes.
• ADVI yields the best forecasting metrics among VI methods in some cases, but FR-ADVI can over- or under-estimate posterior uncertainty compared to NUTS.
• MAP provides high-quality forecasts with fast execution and essentially corresponds to regularized MLE; its uncertainty capture is limited compared to full MCMC.
• VI methods achieve forecasting metrics comparable to full MCMC while significantly reducing computation time; however, FullRank ADVI can overestimate uncertainty and ADVI can underestimate it.
• Summarized results show NUTS achieves convergence with 550 samples and 193.77 seconds, while MH/DMZ require millions of samples and still fail to converge; NUTS offers at least 17% effective samples per total samples.
• Variational methods converge in about 80,000 iterations with roughly 18–23 seconds of runtime depending on method, while MCMC methods span thousands to tens of thousands of seconds depending on method.
• MAP, ADVI, and NUTS each have distinct strengths; the authors suggest NUTS as potentially the best overall choice for this problem given current settings.
• Future work includes GPU acceleration for MCMC (e.g., JAX, NumPyro) and further investigation into posterior uncertainty behavior of ADVI and FR-ADVI.

Figure 2: Autocorrelation plot from NUTS for the slope parameter $k$ in the Linear Trend component