[论文解读] Storage and Memory Characterization of Data Intensive Workloads for Bare Metal Cloud

Hosein Mohammadi Makrani|arXiv (Cornell University)|May 21, 2018

Advanced Data Storage Technologies被引用 6

一句话总结

本文研究了在裸金属云环境中，内存（DRAM容量、频率、通道数）与存储（HDD、SSD SATA、SSD PCIe）对Hadoop、Spark和MPI工作负载的联合影响。通过受控实验发现，I/O密集型工作负载（如Hadoop）对内存参数不敏感，而计算/内存密集型工作负载（如迭代式Spark和MPI任务）则显著受益于高频、多通道DRAM，且SSD可缓解I/O瓶颈，但不会将性能瓶颈转移至内存子系统。

ABSTRACT

As the cost-per-byte of storage systems dramatically decreases, SSDs are finding their ways in emerging cloud infrastructure. Similar trend is happening for main memory subsystem, as advanced DRAM technologies with higher capacity, frequency and number of channels are deploying for cloud-scale solutions specially for non-virtualized environment where cloud subscribers can exactly specify the configuration of underling hardware. Given the performance sensitivity of standard workloads to the memory hierarchy parameters, it is important to understand the role of memory and storage for data intensive workloads. In this paper, we investigate how the choice of DRAM (high-end vs low-end) impacts the performance of Hadoop, Spark, and MPI based Big Data workloads in the presence of different storage types on bare metal cloud. Through a methodical experimental setup, we have analyzed the impact of DRAM capacity, operating frequency, the number of channels, storage type, and scale-out factors on the performance of these popular frameworks. Based on micro-architectural analysis, we classified data-intensive workloads into three groups namely I/O bound, compute bound, and memory bound. The characterization results show that neither DRAM capacity, frequency, nor the number of channels play a significant role on the performance of all studied Hadoop workloads as they are mostly I/O bound. On the other hand, our results reveal that iterative tasks (e.g. machine learning) in Spark and MPI are benefiting from a high-end DRAM in particular high frequency and large number of channels, as they are memory or compute bound. Our results show that using SSD PCIe cannot shift the bottleneck from storage to memory, while it can change the workload behavior from I/O bound to compute bound.

研究动机与目标

• 理解内存与存储参数如何共同影响裸金属云环境中数据密集型工作负载的性能。
• 确定当SSD取代HDD时，性能瓶颈是否会从存储子系统转移至内存子系统。
• 评估DRAM容量、频率及通道数对Hadoop、Spark和MPI框架的影响。
• 通过将工作负载行为分类为I/O密集型、计算密集型或内存密集型，指导大数据集群的硬件选型。
• 基于工作负载类型与性能约束，提供优化服务器配置的设计指南。

提出的方法

• 在裸金属云（IBM/SoftLayer）上开展受控实验，使用Hadoop MapReduce 2.7.1、Spark 2.1.0和MPICH2 3.2。
• 调整关键内存参数：DRAM容量（128GB–512GB）、频率（1866MHz–3200MHz）及通道数（2–4）。
• 在不同横向扩展配置下评估三种存储类型：HDD、SSD SATA和SSD PCIe。
• 基于微观架构分析与性能瓶颈，将工作负载分类为三类——I/O密集型、计算密集型与内存密集型。
• 测量来自BigDataBench与HiBench的多样化工作负载（包括机器学习、图分析与ETL任务）的端到端性能。
• 开发一个实验方程，用于估算在I/O带宽饱和后性能增益开始减弱的核心数阈值。

实验结果

研究问题

• RQ1 在裸金属云环境中，DRAM容量、频率及通道数如何影响Hadoop、Spark与MPI工作负载的性能？
• RQ2 将HDD替换为SSD是否会将性能瓶颈从存储子系统转移至内存子系统？
• RQ3 哪些类型的数据密集型工作负载（如迭代式、I/O密集型）最能从高端内存配置中获益？
• RQ4 I/O带宽在多大程度上限制了在大数据工作负载中增加CPU核心数所带来的性能提升？
• RQ5 工作负载分类（I/O密集型、计算密集型、内存密集型）能否预测其对内存与存储参数的敏感度？

主要发现

• Hadoop工作负载主要为I/O密集型，无论DRAM容量、频率或通道数如何，均未表现出显著性能提升。
• 迭代式Spark与MPI工作负载为内存或计算密集型，采用高频、多通道DRAM后性能最高可提升2.4倍。
• 将HDD替换为SSD可使性能提升1.6倍（Hadoop）、2.4倍（Spark）与3.3倍（MPI），表明存储仍是主要瓶颈。
• 尽管采用SSD，多核CPU的性能仍受I/O带宽限制，核心数超过某一阈值后，性能提升趋于饱和。
• 本研究识别出CPU核心数的性能饱和点，该点可通过基于I/O带宽的实验方程进行预测。
• 结果表明，仅优化内存子系统对I/O密集型工作负载无效，必须先升级存储后，再投资高端内存才具成效。