[Paper Review] Simulation of quasi-stationary distributions on countable spaces

Pablo Groisman, Matthieu Jonckheere|arXiv (Cornell University)|Jun 28, 2012

Stochastic processes and statistical mechanics37 references25 citations

TL;DR

This paper presents a simulation method for quasi-stationary distributions (QSDs) on countable state spaces using Fleming-Viot dynamics and the $μ$-return process. It establishes that fixed points of the map $Φ(\mu)$, which maps a measure $µ$ to the invariant distribution of the $µ$-return process, correspond exactly to QSDs, enabling iterative approximation of QSDs via the dynamics of the return process.

ABSTRACT

Quasi-stationary distributions (QSD) have been widely studied since the pioneering work of Kolmogorov (1938), Yaglom (1947) and Sevastyanov (1951). They appear as a natural object when considering Markov processes that are certainly absorbed since they are invariant for the evolution of the distribution of the process conditioned on not being absorbed. They hence appropriately describe the state of the process at large times for non absorbed paths. Unlike invariant distributions for Markov processes, QSD are solutions of a non-linear equation and there can be 0, 1 or an infinity of them. Also, they cannot be obtained as Cesàro limits of Markovian dynamics. These facts make the computation of QSDs a nontrivial matter. We review different approximation methods for QSD that are useful for simulation purposes, mainly focused on Fleming-Viot dynamics. We also give some alternative proofs and extensions of known results.

Motivation & Objective

• To develop a practical simulation method for quasi-stationary distributions (QSDs) on countable state spaces, where direct simulation is infeasible.
• To establish a connection between the invariant distributions of the $µ$-return process and QSDs of the original absorbing Markov process.
• To provide a theoretical foundation for iterative methods based on the map $Φ(\mu)$, showing convergence to QSDs under appropriate conditions.
• To address open problems in QSD simulation, including convergence rates, higher eigenvectors, and FV dynamics on drifted random walks.

Proposed method

• Define the $µ$-return process, which restarts at $µ$ upon absorption at the absorbing state 0.
• Introduce the map $Φ(\mu)$, which assigns to each initial measure $µ$ the invariant distribution of the $µ$-return process.
• Prove that a measure $ν$ is a QSD if and only if it is a fixed point of $Φ(\mu)$, linking QSD existence to the dynamics of the return process.
• Use the semi-group property of the conditioned evolution $T_t\mu$ to analyze long-time behavior and convergence to QSDs.
• Apply the Kesten-Stigum theorem to multitype branching processes to simulate QSDs in finite state spaces via almost sure convergence of normalized population vectors.
• Leverage Fleming-Viot dynamics as a framework for approximating QSDs, where the empirical measure of $N$ particles converges to a QSD as $N \to \infty$.

Experimental results

Research questions

• RQ1 Under what conditions does the map $\Phi(\mu)$, assigning the invariant measure of the $\mu$-return process, have a fixed point that corresponds to a quasi-stationary distribution?
• RQ2 Can the iterative application of $\Phi^n(\mu)$ converge to a quasi-stationary distribution, and what are the necessary and sufficient conditions for such convergence?
• RQ3 How do Fleming-Viot dynamics with $N$ particles approximate the true quasi-stationary distribution, and what is the rate of convergence as $N \to \infty$?
• RQ4 Is it possible to extend the simulation framework to estimate not only the principal eigenvector (QSD) but also other eigenvectors of the infinitesimal generator?
• RQ5 What are the conditions under which the number of jumps in a Fleming-Viot process driven by a diffusion does not accumulate in finite time intervals?

Key findings

• A measure $\nu$ is a quasi-stationary distribution if and only if it is a fixed point of the map $\Phi(\mu)$, which maps $\mu$ to the invariant distribution of the $\mu$-return process.
• The existence of a QSD is equivalent to the existence of a $\theta > 0$ and $x \in \Lambda$ such that $\mathbb{E}[e^{\theta \tau^x}] < \infty$, under the condition that $P(\tau^x > t) \to 1$ as $x \to \infty$ for every $t > 0$.
• The Kesten-Stigum theorem ensures that in supercritical multitype branching processes, the normalized population vector converges almost surely to a left eigenvector of the mean offspring matrix, which corresponds to the unique QSD.
• For finite state spaces, the normalized population vector of a supercritical multitype branching process converges almost surely to the QSD, providing a practical simulation method.
• The empirical distribution of the Fleming-Viot process with $N$ particles converges to a QSD as $N \to \infty$, with the tagged particle's law converging to $Z^\nu$ in the limit.
• The method based on the $\mu$-return process and the map $\Phi(\mu)$ provides a robust framework for simulating QSDs, especially when combined with Cesàro means or iterative schemes.