[Paper Review] Randomization for Markov chains with applications to networks in a random environment

Ruslan Krenzler, Hans Daduna+1 more|arXiv (Cornell University)|Jul 31, 2014

Advanced Queuing Theory Analysis26 references3 citations

TL;DR

This paper introduces randomized rerouting algorithms for Jacksonian queueing networks operating in a random environment, where service rates fluctuate due to external factors like breakdowns, repairs, or capacity changes. By dynamically adjusting routing probabilities and overall arrival rates, the method maintains network utilization and achieves a product-form stationary distribution for the joint queue length and environment state, enabling efficient performance analysis without explicit computation of the stationary distribution.

ABSTRACT

We develop randomized modifications of Markov chains and apply these modifications to the routing chains of customers in Jacksonian stochastic networks. The aim of our investigations is to find new rerouting schemes for non standard Jackson networks which hitherto resist computing explicitly the stationary distribution. The non standard properties we can handle by suitable algorithms encompass several modifications of Jackson networks known in the literature, especially breakdown and repair of nodes with access modification for customers to down nodes, finite buffers with control of buffer overflow. The rerouting schemes available in the literature for these situations are special cases of our rerouting schemes, which can deal also with partial degrading of service capacities and even with speed up of service. In any case we require our algorithms to react on such general changes in the network with the aim to maintain the utilization of the nodes. To hold this invariant under change of service speeds (intensities) our algorithms not only adapt the routing probabilities but decrease automatically the overall arrival rate to the network if necessary. Our main application is for stochastic networks in a random environment. The impact of the environment on the network is by changing service speeds (by upgrading and/or degrading, breakdown, repair) and we implement the randomization algorithms to react to the changes of the environment. On the other side, customers departing from the network may enforce the environment to jump immediately. So our environment is not Markov for its own. The main result is to compute explicitly the joint stationary distribution of the queue lengths vector and the environment which is of product form: Environment and queue lengths vector, and the queue lengths over the network are decomposable.

Motivation & Objective

• To develop rerouting schemes for non-standard Jackson networks where explicit computation of the stationary distribution is infeasible.
• To maintain network utilization under dynamic changes such as service breakdowns, repairs, degradation, or speed-ups.
• To extend product-form results to networks with non-Markovian environments that are influenced by customer departures.
• To generalize existing rerouting models by incorporating both randomized skipping and reflection mechanisms.
• To provide a unified framework for handling finite buffers, partial capacity degradation, and environment-induced service rate changes.

Proposed method

• Introduces randomized skipping and reflection as two core rerouting mechanisms that adapt routing probabilities based on current service rates and environment states.
• Uses environment-dependent factors γ(k) to model service rate changes (e.g., degradation, upgrade, repair) in the network.
• Defines a joint Markov process (X,Y) for queue lengths X and environment state Y, with transition rates that incorporate both routing and environmental transitions.
• Applies a generalized rerouting scheme r^(α(k)) with invariant measures α_j(k)·η_j to ensure stability and product-form structure.
• Derives the joint stationary distribution as π(n,k) = ξ(n)θ(k), where ξ(n) is the product-form distribution of the pure Jackson network and θ(k) solves a reduced generator equation.
• Establishes that the environment process is not Markov due to two-way interaction with the network, yet the joint process retains product-form stationarity.

Experimental results

Research questions

• RQ1 Can randomized rerouting schemes maintain network utilization under dynamic service rate changes such as breakdowns or upgrades?
• RQ2 Does the joint queue length and environment process admit a product-form stationary distribution despite non-Markovian environmental dynamics?
• RQ3 How can routing be adapted in real time to preserve performance when service capacities degrade or improve?
• RQ4 Can the proposed framework generalize existing models for finite buffers, partial degradation, and repair processes?
• RQ5 What conditions ensure ergodicity and product-form stationarity in networks with environment-dependent routing and service rates?

Key findings

• The joint queue length and environment process (X,Y) is a homogeneous Markov process with a well-defined generator Q^Z that captures both routing and environmental transitions.
• The stationary distribution of the joint process is of product form: π(n,k) = ξ(n)θ(k), where ξ(n) is the standard Jackson network distribution and θ(k) solves a reduced generator equation.
• The environment process is not Markov due to feedback from customer departures, yet the product-form result still holds, extending prior work that required a Markov environment.
• The method automatically reduces the overall arrival rate when service speeds decrease, preserving utilization and stability.
• The framework generalizes existing models, subsuming special cases like finite buffers, node breakdowns, and repair, while maintaining analytical tractability.
• The rerouting schemes via randomized skipping and reflection are shown to be special cases of a broader general randomization framework with provable product-form results.