[Paper Review] Stochastic Simulation Algorithms for Dynamic Probabilistic Networks

Keiji Kanazawa, Daphne Koller+1 more|arXiv (Cornell University)|Feb 20, 2013

Bayesian Modeling and Causal InferenceComputer Science13 references220 citations

TL;DR

This paper introduces two novel stochastic simulation algorithms—evidence reversal (ER) and survival of the fittest (SOF)—for dynamic probabilistic networks (DPNs), which suffer from error accumulation in standard likelihood weighting over time. By restructuring conditional dependencies or reweighting trials based on evidence likelihood, ER/SOF maintains bounded error regardless of time steps, significantly improving accuracy in temporal probabilistic inference.

ABSTRACT

Stochastic simulation algorithms such as likelihood weighting often give fast, accurate approximations to posterior probabilities in probabilistic networks, and are the methods of choice for very large networks. Unfortunately, the special characteristics of dynamic probabilistic networks (DPNs), which are used to represent stochastic temporal processes, mean that standard simulation algorithms perform very poorly. In essence, the simulation trials diverge further and further from reality as the process is observed over time. In this paper, we present simulation algorithms that use the evidence observed at each time step to push the set of trials back towards reality. The first algorithm, "evidence reversal" (ER) restructures each time slice of the DPN so that the evidence nodes for the slice become ancestors of the state variables. The second algorithm, called "survival of the fittest" sampling (SOF), "repopulates" the set of trials at each time step using a stochastic reproduction rate weighted by the likelihood of the evidence according to each trial. We compare the performance of each algorithm with likelihood weighting on the original network, and also investigate the benefits of combining the ER and SOF methods. The ER/SOF combination appears to maintain bounded error independent of the number of time steps in the simulation.

Motivation & Objective

• To address the poor performance of standard stochastic simulation algorithms in dynamic probabilistic networks (DPNs) due to error accumulation over time.
• To develop simulation techniques that maintain accuracy in long-horizon temporal inference by incorporating evidence at each time step.
• To design methods that avoid divergence from reality as simulations progress through time.
• To evaluate the effectiveness of combining evidence reversal and survival of the fittest sampling for improved posterior estimation.
• To demonstrate bounded error independent of the number of time steps in the simulation.

Proposed method

• Evidence reversal (ER) restructures each time slice of a DPN so that evidence nodes become ancestors of state variables, enabling backward propagation of evidence influence.
• Survival of the fittest (SOF) resamples the set of simulation trials at each time step using a stochastic reproduction rate proportional to the likelihood of the observed evidence under each trial.
• The algorithms use likelihood weighting principles but adapt them to the temporal structure of DPNs to reduce error propagation.
• ER modifies the network structure per time slice to allow evidence to influence state variables more directly.
• SOF maintains a diverse set of trials by favoring those consistent with observed evidence, improving estimation accuracy.
• The combination of ER and SOF is shown to maintain bounded error across extended time horizons.

Experimental results

Research questions

• RQ1 Can stochastic simulation in dynamic probabilistic networks maintain accurate posterior estimates over long time sequences?
• RQ2 How can evidence from each time step be effectively used to correct simulation drift in DPNs?
• RQ3 Does restructuring the network or resampling trials based on evidence likelihood improve long-term inference accuracy?
• RQ4 What is the performance of combining evidence reversal and survival of the fittest in reducing error accumulation?
• RQ5 Is the error in posterior estimation bounded over time when using the proposed methods?

Key findings

• The ER/SOF combination maintains bounded error independent of the number of time steps, unlike standard likelihood weighting which accumulates error.
• Evidence reversal significantly improves the accuracy of posterior estimates by enabling backward influence of evidence on state variables.
• Survival of the fittest sampling effectively maintains a representative set of trials by favoring those consistent with observed evidence.
• The combination of ER and SOF outperforms both individual methods and standard likelihood weighting in long-horizon simulations.
• Empirical results show that ER/SOF achieves stable and accurate inference even over extended temporal sequences.
• The proposed methods are effective in dynamic probabilistic networks where standard algorithms fail due to error divergence.