[Paper Review] Identifying Dwarfs Workloads in Big Data Analytics

Wanling Gao, Chunjie Luo+6 more|arXiv (Cornell University)|May 26, 2015

Cloud Computing and Resource Management18 references4 citations

TL;DR

This paper identifies eight fundamental dwarfs workloads—linear algebra, sampling, logic operations, transform operations, set operations, graph operations, statistical operations, and sort—in big data analytics through a multi-domain analysis of 40 representative algorithms. By abstracting recurring computational patterns across search, social networks, e-commerce, bioinformatics, multimedia, and astronomy, the authors establish a minimal, representative benchmark suite that enables efficient, comprehensive evaluation of big data systems.

ABSTRACT

Big data benchmarking is particularly important and provides applicable yardsticks for evaluating booming big data systems. However, wide coverage and great complexity of big data computing impose big challenges on big data benchmarking. How can we construct a benchmark suite using a minimum set of units of computation to represent diversity of big data analytics workloads? Big data dwarfs are abstractions of extracting frequently appearing operations in big data computing. One dwarf represents one unit of computation, and big data workloads are decomposed into one or more dwarfs. Furthermore, dwarfs workloads rather than vast real workloads are more cost-efficient and representative to evaluate big data systems. In this paper, we extensively investigate six most important or emerging application domains i.e. search engine, social network, e-commerce, multimedia, bioinformatics and astronomy. After analyzing forty representative algorithms, we single out eight dwarfs workloads in big data analytics other than OLAP, which are linear algebra, sampling, logic operations, transform operations, set operations, graph operations, statistic operations and sort.

Motivation & Objective

• To address the challenge of benchmarking diverse, complex, and rapidly evolving big data workloads with minimal, representative units of computation.
• To identify a minimal set of computational abstractions—dwarfs—that capture the essential operations across major big data application domains.
• To develop a systematic methodology for extracting frequently occurring computational patterns from real-world algorithms and frameworks.
• To create a representative, cost-efficient benchmark suite (BigDataBench 3.1) based on these dwarfs for evaluating big data systems.
• To provide a foundation for system evaluation, architectural design, and performance optimization in big data analytics.

Proposed method

• Selected six key application domains—search engine, social network, e-commerce, multimedia, bioinformatics, and astronomy—based on coverage and relevance.
• Analyzed 40 representative algorithms from widely used libraries (e.g., Mahout, MLlib), frameworks (e.g., Spark, Hadoop), and benchmarks (e.g., BigBench, AMP Benchmark).
• Extracted frequently occurring computational operations by decomposing algorithms and identifying recurring patterns across diverse data models (structured, semi-structured, unstructured).
• Used a Directed Acyclic Graph (DAG)-like model to represent workloads, with vertices as data subsets and edges as dwarfs, to validate composability and coverage.
• Validated the eight dwarfs by demonstrating their ability to reconstruct the original 40 algorithms through combinations of dwarf operations.
• Developed BigDataBench 3.1 as an open-source benchmark suite incorporating 14 real-world datasets and 33 workloads, grounded in the identified dwarfs.

Experimental results

Research questions

• RQ1 What are the most frequently occurring, fundamental units of computation across diverse big data analytics workloads?
• RQ2 How can a minimal set of computational abstractions (dwarfs) be systematically derived from real-world algorithms and frameworks?
• RQ3 To what extent can the identified dwarfs represent the computational diversity of big data analytics across multiple domains?
• RQ4 How do the proposed dwarfs compare to higher-level abstractions such as the NRC Seven Giants in terms of granularity and representativeness?
• RQ5 Can a benchmark suite based on these dwarfs effectively and efficiently evaluate big data systems without relying on full-scale, complex workloads?

Key findings

• The authors identified eight core dwarfs workloads in big data analytics: linear algebra, sampling, logic operations, transform operations, set operations, graph operations, statistical operations, and sort.
• These dwarfs were derived from a comprehensive analysis of 40 representative algorithms across six major application domains, ensuring broad coverage and representativeness.
• The dwarfs are capable of reconstructing the original 40 algorithms using a DAG-like composition model, demonstrating their expressive power and completeness.
• The dwarfs are at a lower abstraction level than the NRC Seven Giants, focusing on atomic computational units rather than high-level mathematical problems, enabling finer-grained system evaluation.
• The proposed dwarfs cover essential operations in machine learning, data mining, NLP, and graph processing, including key components of optimization, similarity search, and probabilistic inference.
• The resulting BigDataBench 3.1 benchmark suite, publicly available, includes 14 real datasets and 33 workloads, all grounded in the identified dwarfs, offering a scalable and representative evaluation framework.