[Paper Review] Metropolis-type algorithms for Continuous Time Bayesian Networks

Błażej Miasojedow, Wojciech Niemiro+2 more|arXiv (Cornell University)|Mar 17, 2014

Bayesian Modeling and Causal Inference10 references4 citations

TL;DR

This paper proposes a novel Metropolis-Hastings MCMC algorithm for continuous time Bayesian networks (CTBNs) that uses uniformization to represent CTMCs as marked Poisson processes, enabling efficient reversible jump moves for hidden variable detection. The method achieves superior performance over likelihood weighting in high-dimensional and unbounded state spaces, particularly in models like Lotka-Volterra with infinite state spaces.

ABSTRACT

We present a Metropolis-Hastings Markov chain Monte Carlo (MCMC) algorithm for detecting hidden variables in a continuous time Bayesian network (CTBN), which uses reversible jumps in the sense defined by (Green 1995). In common with several Monte Carlo algorithms, one of the most recent and important by (Rao and Teh 2013), our algorithm exploits uniformization techniques under which a continuous time Markov process can be represented as a marked Poisson process. We exploit this in a novel way. We show that our MCMC algorithm can be more efficient than those of likelihood weighting type, as in (Nodelman et al. 2003) and (Fan et al. 2010) and that our algorithm broadens the class of important examples that can be treated effectively.

Motivation & Objective

• To develop an efficient MCMC algorithm for learning hidden variables in continuous time Bayesian networks (CTBNs), especially in cases where existing methods fail.
• To overcome the limitations of likelihood weighting, which suffers from weight degeneracy and poor performance with large or unbounded state spaces.
• To design a reversible jump MCMC algorithm that operates locally and scales independently of state space size, enabling application to models with infinite or very large state spaces.
• To leverage uniformization techniques to represent CTMCs as marked Poisson processes, facilitating the construction of proposal distributions with controlled computational cost.
• To demonstrate the algorithm's effectiveness on complex, realistic models such as the Lotka-Volterra predator-prey system with unbounded state space and non-linear intensities.

Proposed method

• Represents the continuous-time Markov process in a CTBN as a marked Poisson process (MPP) via uniformization, using a rate λ ≥ maxₓ Q(x) to render the process amenable to simulation.
• Constructs Metropolis-Hastings proposals that include reversible jumps—specifically, Add/EraseRandomPoint moves—by modifying the trajectory at a single point while preserving detailed balance.
• Uses local updates that only require recomputing sufficient statistics for three consecutive points in the trajectory, minimizing computational overhead per iteration.
• Employs a proposal mechanism where new states are generated based on the transition kernel from the previous state, with acceptance probability derived from the Metropolis-Hastings ratio.
• Applies the algorithm to CTBNs with hidden variables, where only a subset of variables are observed, and uses the full trajectory to infer posterior distributions over unobserved states.
• Tunes the uniformization rate λ to balance acceptance probability and computational cost, avoiding both low acceptance (when λ ≈ max Q(x)) and excessive virtual jumps (when λ is too large).

Experimental results

Research questions

• RQ1 Can a Metropolis-Hastings MCMC algorithm be designed for CTBNs that efficiently handles models with unbounded state spaces, where standard methods fail?
• RQ2 How can reversible jump MCMC be adapted to continuous-time processes using uniformization, ensuring detailed balance and efficient exploration of the state space?
• RQ3 What is the performance trade-off between acceptance rate and computational cost when choosing the uniformization rate λ in the proposed algorithm?
• RQ4 How does the proposed MCMC method compare to likelihood weighting and other MCMC approaches (e.g., Rao and Teh 2013) in terms of convergence and accuracy on complex models?
• RQ5 Can the algorithm effectively infer hidden variables in a CTBN when the observed data consists of sampled trajectories at fixed time points rather than full paths?

Key findings

• The proposed MCMC algorithm achieves an acceptance rate of approximately 0.5 in experiments, indicating good mixing and efficient exploration of the state space.
• The algorithm significantly outperforms likelihood weighting in terms of estimation accuracy and stability, especially in high-dimensional or unbounded state space models.
• In the Lotka-Volterra example with unbounded state space and non-linear intensities, the algorithm successfully recovered posterior medians and means with narrow credible intervals, demonstrating robustness.
• The computational cost of the algorithm is largely independent of the size of the state space, making it suitable for models with infinite or very large state spaces.
• The algorithm's local update mechanism—requiring only three-point sufficient statistics—results in lower per-iteration cost compared to global re-evaluation methods like those in Rao and Teh (2013).
• The method remains effective even when intensities are unbounded (as long as they are locally bounded for uniformization), showing that truncation is not required for practical application.