[Paper Review] Metastability of finite state Markov chains: a recursive procedure to identify slow variables for model reduction

Cláudio Landim, Tengfei Xu|arXiv (Cornell University)|Dec 21, 2015

Markov Chains and Monte Carlo Methods12 references3 citations

TL;DR

This paper presents a recursive algorithm to identify slow variables in finite-state, continuous-time Markov chains by constructing a hierarchy of increasingly coarse partitions of the state space and corresponding time-scales. The method ensures that the projected dynamics on each partition converge to a Markov chain, enabling effective model reduction while preserving metastable behavior, with convergence proven under a logarithmic rate ratio condition.

ABSTRACT

Consider a sequence $(η^N(t) :t\ge 0)$ of continuous-time, irreducible Markov chains evolving on a fixed finite set $E$, indexed by a parameter $N$. Denote by $R_N(η,ξ)$ the jump rates of the Markov chain $η^N_t$, and assume that for any pair of bonds $(η,ξ)$, $(η',ξ')$ $\arctan \{R_N(η,ξ)/R_N(η',ξ')\}$ converges as $N\uparrow\infty$. Under a hypothesis slightly more restrictive (cf. \eqref{mhyp} below), we present a recursive procedure which provides a sequence of increasing time-scales $θ^1_N, \dots, θ^{\mf p}_N$, $θ^j_N \ll θ^{j+1}_N$, and of coarsening partitions $\{\ms E^j_1, \dots, \ms E^j_{\mf n_j}, Δ^j\}$, $1\le j\le \mf p$, of the set $E$. Let $ϕ_j: E o \{0,1, \dots, \mf n_j\}$ be the projection defined by $ ϕ_j(η) = \sum_{x=1}^{\mf n_j} x \, \mb 1\{η\in \ms E^j_x\}$. For each $1\le j\le \mf p$, we prove that the hidden Markov chain $X^j_N(t) = ϕ_j(η^N(tθ^j_N))$ converges to a Markov chain on $\{1, \dots, \mf n_j\}$.

Motivation & Objective

• To develop a systematic method for identifying slow variables in finite-state, continuous-time Markov chains that enable model reduction.
• To construct a hierarchy of increasingly coarse partitions of the state space, each associated with a distinct time-scale.
• To ensure that the dynamics of the projected chain on each partition converge to a Markov process in the corresponding time-scale.
• To provide a recursive framework that reveals metastable structures through successive coarsening of the state space.
• To formalize conditions under which the time spent in separating sets (Δj) becomes negligible in the limit.

Proposed method

• The method recursively constructs time-scales θjN and partitions {Ej1,…,Ejnj,Δj} for j=1,…,p, with increasing coarseness and increasing Δj.
• It relies on the assumption that arctan(RN(η,ξ)/RN(η′,ξ′)) converges as N→∞, ensuring logarithmic rate ratios stabilize.
• For each level j, the projection φj(η)=∑x=1nj x⋅1{η∈Ejx} defines a slow variable evolving on time-scale θjN.
• The convergence of XjN(t)=φj(ηN(tθjN)) to a Markov chain on {1,…,nj} is proven via convergence in Skorohod topology.
• The procedure uses the trace process ηFt on subsets F⊂E to analyze hitting times and recurrence, ensuring negligible time in Δj.
• Key technical tools include the strong Markov property, ergodicity of trace processes, and convergence of rescaled hitting times to exponential laws.

Experimental results

Research questions

• RQ1 How can one systematically identify slow variables in finite-state, continuous-time Markov chains exhibiting metastable behavior?
• RQ2 What recursive procedure ensures that the dynamics on coarsened partitions converge to a Markov process across multiple time-scales?
• RQ3 Under what conditions does the time spent in the separating set Δj become asymptotically negligible in the time-scale θjN?
• RQ4 How can the hierarchy of partitions and time-scales be constructed so that each level reveals a coarser, yet Markovian, effective dynamics?
• RQ5 What mathematical conditions guarantee that the chain visits all states within a valley before jumping to another?

Key findings

• The projected process XjN(t)=φj(ηN(tθjN)) converges in law to a continuous-time, irreducible Markov chain on {1,…,nj} as N→∞.
• The time-scale hierarchy satisfies θjN≪θj+1N, ensuring separation of time-scales across successive levels of coarse-graining.
• The set Δj increases with j, and the time spent in Δj during the θjN time-scale vanishes in the limit, as formalized in condition (H3).
• For any initial state in Ejx, the hitting time to the union of other Ejy sets is asymptotically exponentially distributed in the θjN time-scale.
• With probability tending to 1, the chain visits all states in Ejx before exiting to another Ejy, confirming internal ergodicity of each valley.
• The recursive construction ensures that each level j yields a valid, asymptotically Markovian, reduced model on a smaller state space.