[Paper Review] Analysis of structured Markov processes

Ivo Adan, Johan S. H. van Leeuwaarden+1 more|arXiv (Cornell University)|Sep 26, 2017

Advanced Queuing Theory Analysis11 references3 citations

TL;DR

This paper presents a comprehensive framework for modeling and analyzing structured Markov processes using systematic classification based on structural properties. It introduces foundational analytic methods—such as matrix-geometric, matrix-analytic, and spectral expansion techniques—enabling efficient solution of complex queueing and stochastic systems, with key contributions in stability analysis, transform inversion, and numerical algorithms applicable across engineering and finance.

ABSTRACT

Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We classify Markov processes based on their structural properties, which in turn determine which analytic methods are required for solving them. In doing so, we start in each chapter with specific examples that naturally lead up to general theory and general methods. In this way the reader learns about Markov processes on the job. By studying this book, the reader becomes acquainted with the basic analytic methods that come into play when systems are modeled as structured Markov processes. These basic methods will likely prove useful, in real-time when studying the examples at hand, but more importantly for future encounters with Markov processes not covered in this book. Methods are more important than examples. The methods have a large scope of application, even outside the scope of Markov processes, in areas like probability theory, industrial engineering, mechanical engineering, physics and financial mathematics.

Motivation & Objective

• To develop a unified methodology for analyzing structured Markov processes based on their intrinsic structural properties.
• To bridge theoretical Markov process analysis with practical applications in queueing, manufacturing, and service systems.
• To equip researchers and practitioners with transferable analytic tools applicable beyond Markov processes, including in probability, physics, and financial mathematics.
• To provide a graduate-level resource integrating theory, examples, and numerical methods for solving complex stochastic models.
• To establish robust computational techniques—such as matrix-geometric and spectral expansion methods—for equilibrium and transient analysis.

Proposed method

• Classifies Markov processes by structural features (e.g., birth-and-death, quasi-birth-and-death, skip-free) to determine appropriate analytic techniques.
• Applies transforms (Laplace, generating functions) to derive equilibrium and transient distributions, with numerical inversion for practical evaluation.
• Employs the matrix-geometric method to solve QBD and quasi-skip-free processes by exploiting block structure in transition matrices.
• Utilizes the matrix-analytic method to compute stationary distributions via matrix roots and iterative algorithms.
• Introduces the spectral expansion method to decompose solutions into eigenmodes for systems with infinite state spaces.
• Applies compensation and difference equation approaches for priority and gated service systems, enabling exact solution derivation.

Experimental results

Research questions

• RQ1 How can Markov processes be systematically classified by structural properties to guide method selection?
• RQ2 What analytic techniques are most effective for solving structured Markov processes such as QBD and quasi-skip-free processes?
• RQ3 How can transforms and their numerical inversion be used to compute equilibrium and transient distributions efficiently?
• RQ4 What are the stability conditions for quasi-birth-and-death and quasi-skip-free processes, and how do they affect system behavior?
• RQ5 In what ways do matrix-analytic and spectral expansion methods outperform traditional approaches in solving complex queueing systems?

Key findings

• The matrix-geometric method enables efficient computation of stationary distributions for quasi-birth-and-death processes by exploiting block-Toeplitz structure in transition matrices.
• Stability conditions for QBD and quasi-skip-free processes are derived through spectral analysis of the rate matrix, ensuring positive recurrence.
• Numerical inversion of transforms allows accurate and efficient evaluation of transient and equilibrium distributions in complex systems.
• The spectral expansion method provides a convergent representation of solutions for infinite-state systems, with explicit error bounds in certain cases.
• The compensation approach successfully solves gated and priority queueing systems by iteratively balancing flow equations across state blocks.
• The matrix-analytic method generalizes to a wide class of structured processes, offering a unifying framework for equilibrium analysis across diverse applications.