[Paper Review] Gibbs Sampling in Factorized Continuous-Time Markov Processes

Tal El‐Hay, Nir Friedman+1 more|arXiv (Cornell University)|Jun 13, 2012

Gaussian Processes and Bayesian InferenceComputer Science14 references56 citations

TL;DR

This paper introduces a novel Gibbs sampling procedure for factorized continuous-time Markov processes, enabling asymptotically unbiased inference in multi-component systems. By iteratively resampling trajectories of individual components while conditioning on others, and leveraging exact sampling adapted to each component's natural time scale, the method reduces computational cost through structural exploitation of the network, offering the first such approach with guaranteed asymptotic unbiasedness in this setting.

ABSTRACT

A central task in many applications is reasoning about processes that change over continuous time. Continuous-Time Bayesian Networks is a general compact representation language for multi-component continuous-time processes. However, exact inference in such processes is exponential in the number of components, and thus infeasible for most models of interest. Here we develop a novel Gibbs sampling procedure for multi-component processes. This procedure iteratively samples a trajectory for one of the components given the remaining ones. We show how to perform exact sampling that adapts to the natural time scale of the sampled process. Moreover, we show that this sampling procedure naturally exploits the structure of the network to reduce the computational cost of each step. This procedure is the first that can provide asymptotically unbiased approximation in such processes.

Motivation & Objective

• To address the intractability of exact inference in continuous-time Bayesian networks due to exponential complexity in component count.
• To develop a sampling-based inference method that is asymptotically unbiased for multi-component continuous-time processes.
• To exploit structural dependencies in the network to reduce computational cost per sampling step.
• To enable efficient inference by adapting sampling to the natural time scale of each component.
• To provide a practical, scalable alternative to exact inference in complex continuous-time stochastic processes.

Proposed method

• The method performs Gibbs sampling by iteratively resampling the trajectory of one component while conditioning on the current trajectories of all other components.
• For each component, exact sampling is performed using its conditional intensity functions, adapted to the component’s intrinsic time scale.
• The algorithm exploits conditional independence structures in the factorized network to limit the scope of computation per sampling step.
• Trajectory generation is based on stochastic simulation techniques tailored to continuous-time jump processes, ensuring correctness under the conditional distribution.
• The procedure maintains detailed balance, ensuring convergence to the true posterior distribution over trajectories.
• Time-scale adaptation allows efficient sampling by focusing on relevant time intervals for each component.

Experimental results

Research questions

• RQ1 Can a Gibbs sampling procedure be designed to provide asymptotically unbiased inference in factorized continuous-time Markov processes?
• RQ2 How can exact sampling be efficiently performed for individual components while respecting their natural time scales?
• RQ3 In what way can the conditional independence structure of the network be exploited to reduce computational cost per sampling step?
• RQ4 What is the impact of time-scale adaptation on the mixing and convergence rate of the Markov chain?
• RQ5 Can this method scale to models with many components where exact inference is infeasible?

Key findings

• The proposed Gibbs sampling procedure achieves asymptotically unbiased inference, providing consistent estimates as the number of samples increases.
• By adapting sampling to each component’s natural time scale, the method significantly improves sampling efficiency and reduces computational overhead.
• The algorithm exploits conditional independence in the network structure, leading to reduced per-step computation compared to full-process sampling.
• The method enables practical inference in complex continuous-time Bayesian networks where exact inference is computationally infeasible.
• Empirical results demonstrate improved mixing and faster convergence compared to naive sampling approaches.
• The approach is the first to combine exact component-wise sampling with structural exploitation in continuous-time factorized processes.