[Paper Review] Iterative trajectory reweighting for estimation of equilibrium and non-equilibrium observables

John D. Russo, Jeremy Copperman+1 more|arXiv (Cornell University)|Jun 16, 2020

Spectroscopy and Quantum Chemical Studies4 citations

TL;DR

This paper introduces iterative trajectory reweighting algorithms that estimate equilibrium and non-equilibrium observables—such as stationary distributions, committors, and mean first-passage times—without assuming Markovianity or requiring transition matrices. By leveraging the stationarity of probability distributions and committor values across continuous trajectories, the method converges to unbiased, self-consistent estimates through iterative refinement of trajectory weights based on time-averaged bin probabilities or downstream committor values.

ABSTRACT

We present two algorithms by which a set of short, unbiased trajectories can be iteratively reweighted to obtain various observables. The first algorithm estimates the stationary (steady state) distribution of a system by iteratively reweighting the trajectories based on the average probability in each state. The algorithm applies to equilibrium or non-equilibrium steady states, exploiting the `left' stationarity of the distribution under dynamics -- i.e., in a discrete setting, when the column vector of probabilities is multiplied by the transition matrix expressed as a left stochastic matrix. The second procedure relies on the `right' stationarity of the committor (splitting probability) expressed as a row vector. The algorithms are unbiased, do not rely on computing transition matrices, and make no Markov assumption about discretized states. Here, we apply the procedures to a one-dimensional double-well potential, and to a 208$\\mu$s atomistic Trp-cage folding trajectory from D.E. Shaw Research.

Motivation & Objective

• Address the challenge of limited sampling timescales in molecular dynamics simulations, particularly for rare events and long-time dynamics.
• Overcome limitations of Markov state models (MSMs) that rely on Markovian assumptions and lag-time selection.
• Develop a method to estimate equilibrium and non-equilibrium observables—like stationary distributions, committors, and mean first-passage times—from short, unbiased trajectories.
• Enable estimation of observables without constructing transition matrices or discretizing dynamics into Markov chains.
• Provide a self-consistent, iterative framework that leverages stationarity of probability distributions and committor values in continuous trajectory data.

Proposed method

• Use iterative reweighting of trajectories based on time-averaged probabilities in discrete phase-space bins to estimate the stationary (equilibrium or non-equilibrium steady-state) distribution.
• Apply left stationarity of the probability vector under dynamics: the distribution remains invariant when multiplied by the transition matrix, enabling self-consistent weight updates.
• For committor estimation, exploit right stationarity of the committor vector: the average committor of trajectories downstream from a bin converges to the bin’s committor value.
• Initialize trajectory weights uniformly and iteratively update them using time-averaged probabilities from the prior iteration, ensuring convergence to a stationary solution.
• Apply boundary conditions: committor values are set to 1 for trajectories reaching the target macrostate and 0 for the off-target state.
• Use continuous trajectories as the sole input, preserving all dynamical details without assuming Markovian transitions between discretized states.

Experimental results

Research questions

• RQ1 Can we estimate the equilibrium distribution of a system from short, unbiased trajectories without assuming Markovian dynamics or constructing transition matrices?
• RQ2 Can the method reliably estimate non-equilibrium steady-state (NESS) distributions when external or boundary conditions are properly accounted for in trajectory preparation?
• RQ3 How accurately can the iterative reweighting approach estimate the committor (splitting probability) for complex systems like the Trp-cage protein, compared to brute-force reference calculations?
• RQ4 Does the iterative procedure converge to an unbiased estimate of mean first-passage times (MFPT) via the Hill relation, using only trajectory data?
• RQ5 To what extent does the method remain robust and accurate when phase space is discretized into bins, despite the absence of a Markov assumption?

Key findings

• The iterative equilibrium distribution estimator converges to the exact Boltzmann distribution in the 1D double-well potential system, with no bias observed even after one iteration.
• For the Trp-cage folding trajectory, the iterative method produces equilibrium distributions that agree well with simple bin counts, though the rightmost bin shows notable deviation requiring further investigation.
• The iterative committor estimator produces results that closely track brute-force reference values in the 1D system, with a sigmoid-shaped profile matching theoretical expectations.
• In the Trp-cage system, the iterative committor estimates show strong agreement with brute-force estimates, particularly near the macrostates, with a correlation close to 1 across independent trials.
• The method achieves convergence without requiring transition matrices or Markov assumptions, relying only on continuous trajectories and iterative self-consistency.
• The approach is formally equivalent to the power method applied to a non-standard transition matrix that captures full dynamics across all timescales, as confirmed by theoretical insights from collaborators.