[Paper Review] Open Markov Processes and Reaction Networks

Blake S. Pollard|arXiv (Cornell University)|Sep 27, 2017

Gene Regulatory Network Analysis3 citations

TL;DR

This dissertation develops a categorical framework for open Markov processes and reaction networks, showing that non-equilibrium steady states minimize a quadratic dissipation function—closely related to Prigogine's minimum entropy production principle. It introduces a black-boxing functor that maps open systems to linear relations of boundary probabilities and flows, enabling compositional analysis of complex systems via category theory and gradient flow structures.

ABSTRACT

We define the concept of an `open' Markov process, a continuous-time Markov chain equipped with specified boundary states through which probability can flow in and out of the system. External couplings which fix the probabilities of boundary states induce non-equilibrium steady states characterized by non-zero probability currents flowing through the system. We show that these non-equilibrium steady states minimize a quadratic form which we call `dissipation.' This is closely related to Prigogine's principle of minimum entropy production. We bound the rate of change of the entropy of a driven non-equilibrium steady state relative to the underlying equilibrium state in terms of the flow of probability through the boundary of the process. We then consider open Markov processes as morphisms in a symmetric monoidal category by splitting up their boundary states into certain sets of `inputs' and `outputs.' Composition corresponds to gluing the outputs of one such open Markov process onto the inputs of another so that the probability flowing out of the first process is equal to the probability flowing into the second. We construct a `black-box' functor characterizing the behavior of an open Markov process in terms of the space of possible steady state probabilities and probability currents along the boundary. The fact that this is a functor means that the behavior of a composite open Markov process can be computed by composing the behaviors of the open Markov processes from which it is composed. We prove a similar black-boxing theorem for reaction networks whose dynamics are given by the non-linear rate equation. Along the way we describe a more general category of open dynamical systems where composition corresponds to gluing together open dynamical systems.

Motivation & Objective

• To establish a compositional framework for open Markov processes using category theory, enabling modular analysis of non-equilibrium systems.
• To demonstrate that non-equilibrium steady states in open Markov processes minimize a quadratic dissipation function, linking to Prigogine's principle of minimum entropy production.
• To construct a black-boxing functor that maps open Markov processes to linear relations of boundary probabilities and currents, preserving compositionality.
• To extend the framework to reaction networks by showing that steady-state behaviors can be captured via semialgebraic relations, generalizing the compositional approach.
• To explore the potential for a similar variational principle in complex balanced reaction networks, linking free energy and gradient flow structures.

Proposed method

• Define open Markov processes as continuous-time Markov chains with boundary states where probability flows in and out.
• Use the master equation to model time evolution, with detailed balanced equilibria satisfying detailed balance conditions.
• Introduce the concept of 'dissipation' as a quadratic form minimized by non-equilibrium steady states under external boundary conditions.
• Construct a symmetric monoidal category of open Markov processes, where composition corresponds to gluing outputs to inputs and tensoring to parallel composition.
• Define a black-boxing functor from open Markov processes to the category of linear relations (LinRel), mapping each system to its boundary behavior (probabilities and currents).
• Leverage decorated cospans and functorial composition to generalize the framework to open dynamical systems and reaction networks with non-linear rate equations.

Experimental results

Research questions

• RQ1 How can open Markov processes be composed in a way that preserves their dynamical behavior, and what categorical structure supports this?
• RQ2 What variational principle governs non-equilibrium steady states in open Markov processes, and how is it related to entropy production and dissipation?
• RQ3 Can the behavior of open reaction networks be systematically described via a compositional black-boxing functor, and how does this relate to steady-state concentrations and flows?
• RQ4 What is the relationship between relative entropy, free energy, and dissipation in non-equilibrium Markov processes near equilibrium?
• RQ5 Is there a natural categorical framework for complex balanced reaction networks that supports a variational principle analogous to the one for detailed balanced Markov processes?

Key findings

• Non-equilibrium steady states in open Markov processes minimize a quadratic dissipation function, which approximates the rate of entropy production for states near equilibrium.
• The black-boxing functor maps each open Markov process to a linear subspace of boundary probabilities and currents, ensuring that the behavior of a composite system is the composition of its parts.
• For detailed balanced Markov processes, the steady-state behavior can be mapped to electrical circuits via a functor K, and the black-boxing of such systems yields a linear relation in LinRel.
• Relative entropy serves as a Lyapunov function for Markov processes, decreasing over time and approximating dissipation in the near-equilibrium regime.
• The open master equation can be expressed as a gradient flow with respect to the dissipation function, where the gradient is defined via a metric involving the equilibrium distribution.
• A general framework for open reaction networks is established via a functor to semialgebraic relations, extending the compositional approach beyond linear systems to non-linear rate equations.