[Paper Review] Automatic Generation of Probabilistic Programming from Time Series Data

Anh Tong, Jaesik Choi|arXiv (Cornell University)|Jul 4, 2016

Time Series Analysis and Forecasting12 references5 citations

TL;DR

This paper proposes StanABCD, a system that automatically generates Stan probabilistic programming code from time series data by leveraging the Automatic Bayesian Covariance Discovery (ABCD) framework. It translates descriptive covariance structures—learned via compositional Gaussian processes—into executable Stan code, enabling efficient Bayesian inference and extrapolation with high fidelity to ABCD's results.

ABSTRACT

Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of model is given, constructing a probabilistic program is rather straightforward. Thus, main focus have been to learn the best model parameters and compute marginal probabilities. In this paper, we provide a new perspective to build expressive probabilistic program from continue time series data when the structure of model is not given. The intuition behind of our method is to find a descriptive covariance structure of time series data in nonparametric Gaussian process regression. We report that such descriptive covariance structure efficiently derives a probabilistic programming description accurately.

Motivation & Objective

• To automate the generation of probabilistic programming code from time series data without requiring manual model specification.
• To bridge the gap between automatic statistical modeling (ABCD) and executable probabilistic programming (Stan) for scalable inference.
• To enable reliable, one-to-one mapping from ABCD's learned kernel structures to Stan code for downstream statistical analysis.
• To accelerate kernel learning in ABCD by offloading inference to a compiled probabilistic programming environment.
• To support complex modeling of heterogeneous time series through compact, interpretable Stan code representations.

Proposed method

• Extract descriptive covariance structures from time series using ABCD, which identifies interpretable components like smoothness, periodicity, and changepoints.
• Represent the learned composite kernel as a sum and product of base kernels (e.g., SE, PER, WN, LIN, CONST) using compositional kernel learning.
• Map the composite kernel to Stan syntax by translating kernel operations into matrix operations: addition as matrix sum, multiplication as Hadamard product.
• Implement the joint Gaussian process prior in Stan’s transformed parameters block using Cholesky decomposition for efficient multivariate normal sampling.
• Use the generated quantities block to perform linear transformation of standard normal variates into predictive outputs via mean and Cholesky-decomposed covariance.
• Compile and execute the generated Stan code using PyStan to sample from the posterior predictive distribution on test points.

Experimental results

Research questions

• RQ1 Can ABCD’s learned covariance structures be systematically translated into executable probabilistic programs in Stan?
• RQ2 How accurately can the generated Stan code reproduce the extrapolation performance of ABCD on real-world time series?
• RQ3 To what extent does the automatic code generation preserve the interpretability and expressiveness of the original ABCD model?
• RQ4 Can the generated Stan programs enable efficient Bayesian inference and posterior sampling for new data points?
• RQ5 What is the fidelity of the generated probabilistic programs compared to the original ABCD model in terms of predictive uncertainty and functional form?

Key findings

• StanABCD successfully generates valid, executable Stan code from ABCD’s learned composite kernel structures with a one-to-one mapping.
• The generated Stan programs produce extrapolation samples that closely match the performance of ABCD on both the airline passenger and Indonesian Rupiah exchange rate datasets.
• The method preserves the interpretability of ABCD’s model components (e.g., periodicity, linear trends, noise) in the resulting probabilistic program.
• The use of Cholesky decomposition in the transformed parameters block enables efficient and numerically stable sampling from the joint Gaussian process prior.
• The system demonstrates that automatic probabilistic programming generation is feasible and accurate, enabling scalable inference without manual model re-implementation.
• The generated code supports downstream tasks such as posterior predictive sampling and uncertainty quantification with minimal engineering effort.