[论文解读] Scout: An Experienced Guide to Find the Best Cloud Configuration

Chin-Jung Hsu, Vivek Nair+2 more|arXiv (Cornell University)|Mar 4, 2018

Cloud Computing and Resource Management参考文献 18被引用 42

一句话总结

Scout 使用历史低级别性能数据和对偶学习来引导基于搜索的优化，在比先前方法更低的搜索成本下实现接近最佳的云配置。

ABSTRACT

Finding the right cloud configuration for workloads is an essential step to ensure good performance and contain running costs. A poor choice of cloud configuration decreases application performance and increases running cost significantly. While Bayesian Optimization is effective and applicable to any workloads, it is fragile because performance and workload are hard to model (to predict). In this paper, we propose a novel method, SCOUT. The central insight of SCOUT is that using prior measurements, even those for different workloads, improves search performance and reduces search cost. At its core, SCOUT extracts search hints (inference of resource requirements) from low-level performance metrics. Such hints enable SCOUT to navigate through the search space more efficiently---only spotlight region will be searched. We evaluate SCOUT with 107 workloads on Apache Hadoop and Spark. The experimental results demonstrate that our approach finds better cloud configurations with a lower search cost than state of the art methods. Based on this work, we conclude that (i) low-level performance information is necessary for finding the right cloud configuration in an effective, efficient and reliable way, and (ii) a search method can be guided by historical data, thereby reducing cost and improving performance.

研究动机与目标

• Motivate the problem of selecting optimal cloud configurations for workloads to balance performance and cost.
• Propose Scout as a history-informed, search-based method that improves exploration/exploitation trade-offs.
• Show that low-level performance metrics and transfer learning can guide efficient configuration search.
• Demonstrate scalability and reliability across a large set of workloads and configurations.

提出的方法

• Formulate cloud configuration search as a sequential optimization over a fixed configuration space.
• Extract search hints from low-level performance metrics to guide a pairwise comparison model.
• Use historical data from previous workloads to inform a relaxed, transfer-learned model for ranking configurations.
• Adopt a pairwise (relative) prediction approach instead of predicting absolute performance.
• Employ a search strategy that selects the next configuration with the highest predicted probability of being better than the current best, leveraging history for faster convergence.
• Provide a stopping criterion based on probability threshold and misprediction tolerance.

实验结果

研究问题

• RQ1 Can historical performance data from other workloads reduce the exploration cost in finding near-optimal cloud configurations?
• RQ2 Does a pairwise, transfer-learned, low-level-metric-based model improve search accuracy and convergence speed compared to prior methods like CherryPick and PARIS?
• RQ3 How do low-level metrics and transfer learning affect reliability across diverse workloads?
• RQ4 What are Scout’s performance and cost trade-offs across a large set of workloads and cloud configurations?
• RQ5 Is the approach robust to initial points and parameter settings?

主要发现

• Scout finds near-optimal configurations (within 10%) for 87% of 107 workloads in single-node experiments.
• Scout achieves lower search costs than CherryPick and random baselines across execution time and deployment cost optimization.
• Using historical data and low-level metrics yields more reliable and faster convergence than prior methods.
• The approach shows lower variance across runs, indicating improved reliability over competing methods.
• Cost optimization is harder and often requires more search steps than time optimization, but Scout still converges efficiently.
• The evaluation includes 18 workloads on 69 cloud configurations in multi-node settings, demonstrating scalability.