[Paper Review] Dynamic Virtual Machine Management via Approximate Markov Decision Process

Zhenhua Han, Haisheng Tan+4 more|arXiv (Cornell University)|Jan 30, 2016

Cloud Computing and Resource Management17 references4 citations

TL;DR

This paper proposes MadVM, an approximate Markov Decision Process (MDP)-based dynamic virtual machine (VM) management framework that optimizes energy efficiency and resource utilization in data centers. By modeling VM resource demands as a Markov chain and using value iteration with convergence guarantees, MadVM reduces power consumption by up to 47% and resource shortages to near zero, outperforming baseline methods like CompVM and CloudScale on real-world traces.

ABSTRACT

Efficient virtual machine (VM) management can dramatically reduce energy consumption in data centers. Existing VM management algorithms fall into two categories based on whether the VMs' resource demands are assumed to be static or dynamic. The former category fails to maximize the resource utilization as they cannot adapt to the dynamic nature of VMs' resource demands. Most approaches in the latter category are heuristical and lack theoretical performance guarantees. In this work, we formulate dynamic VM management as a large-scale Markov Decision Process (MDP) problem and derive an optimal solution. Our analysis of real-world data traces supports our choice of the modeling approach. However, solving the large-scale MDP problem suffers from the curse of dimensionality. Therefore, we further exploit the special structure of the problem and propose an approximate MDP-based dynamic VM management method, called MadVM. We prove the convergence of MadVM and analyze the bound of its approximation error. Moreover, MadVM can be implemented in a distributed system, which should suit the needs of real data centers. Extensive simulations based on two real-world workload traces show that MadVM achieves significant performance gains over two existing baseline approaches in power consumption, resource shortage and the number of VM migrations. Specifically, the more intensely the resource demands fluctuate, the more MadVM outperforms.

Motivation & Objective

• To address the limitations of static VM management, which fails to adapt to dynamic resource demands and underutilizes resources.
• To overcome the lack of theoretical guarantees in existing heuristic-based dynamic VM management approaches.
• To formulate dynamic VM management as a large-scale infinite-horizon MDP problem minimizing long-term cost (energy + resource shortage).
• To design a scalable, distributed, and provably convergent algorithm that approximates the optimal MDP solution.
• To evaluate the method on real-world workload traces and demonstrate significant performance gains in power efficiency, resource utilization, and migration overhead.

Proposed method

• Formulate dynamic VM management as an infinite-horizon MDP with a cost function combining power consumption and resource shortage.
• Model VM resource demand transitions using a first-order Markov chain, validated by real-world trace analysis showing quasi-stationary and non-uniform transition probabilities.
• Propose MadVM, an approximate MDP algorithm based on value iteration with a contraction mapping to ensure convergence.
• Introduce a distributed implementation that requires minimal global state, enhancing scalability and fault tolerance.
• Prove convergence of MadVM and derive an upper bound on the approximation error relative to the optimal MDP solution.
• Use a modified value iteration scheme with a projection step onto a low-dimensional subspace to reduce computational complexity.

Experimental results

Research questions

• RQ1 Can modeling VM resource demand dynamics as a Markov chain effectively capture temporal correlations in real-world data center workloads?
• RQ2 How can the curse of dimensionality in large-scale MDPs be mitigated while preserving theoretical performance guarantees in dynamic VM management?
• RQ3 Can an approximate MDP-based approach outperform existing heuristic-based VM management algorithms in terms of energy efficiency and resource availability?
• RQ4 What is the approximation error bound of the proposed algorithm relative to the optimal MDP solution?
• RQ5 Can the algorithm be efficiently implemented in a distributed manner without sacrificing convergence or performance?

Key findings

• MadVM reduces average power consumption by 19% and 42% compared to CompVM and CloudScale, respectively, on real-world workload traces.
• In high-fluctuation workloads, MadVM achieves up to 47% lower power consumption than CloudScale, demonstrating superior adaptability.
• MadVM maintains near-zero resource shortages across all evaluated workloads, significantly outperforming baselines in service-level agreement compliance.
• The algorithm achieves convergence with a bounded approximation error, proven via contraction mapping and sequence analysis.
• The distributed design of MadVM enables robustness and scalability, suitable for large-scale data center deployments.
• Empirical results confirm that modeling VM demand as a Markov chain with quasi-static transition probabilities is effective for dynamic VM management.