[Paper Review] Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data

Yunhong Gu, Robert L. Grossman|ArXiv.org|Sep 6, 2008

Cloud Computing and Resource Management8 references11 citations

TL;DR

This paper introduces Sector and Sphere, a distributed storage and computing system designed to simplify large-scale data processing across geographically dispersed data centers. By enabling unified management of data and user-defined functions (UDFs) across data centers, the system achieves ~2x faster performance than Hadoop on the Terasort benchmark, demonstrating improved scalability and efficiency for cloud-scale workloads.

ABSTRACT

Cloud computing has demonstrated that processing very large datasets over commodity clusters can be done simply given the right programming model and infrastructure. In this paper, we describe the design and implementation of the Sector storage cloud and the Sphere compute cloud. In contrast to existing storage and compute clouds, Sector can manage data not only within a data center, but also across geographically distributed data centers. Similarly, the Sphere compute cloud supports User Defined Functions (UDF) over data both within a data center and across data centers. As a special case, MapReduce style programming can be implemented in Sphere by using a Map UDF followed by a Reduce UDF. We describe some experimental studies comparing Sector/Sphere and Hadoop using the Terasort Benchmark. In these studies, Sector is about twice as fast as Hadoop. Sector/Sphere is open source.

Motivation & Objective

• Address the complexity of managing and processing large-scale data across multiple, geographically distributed data centers.
• Overcome limitations of existing storage and compute clouds that primarily operate within a single data center.
• Provide a unified programming model that supports both intra- and inter-data center data processing using User Defined Functions (UDFs).
• Enable efficient implementation of MapReduce-style workloads through modular Map and Reduce UDFs.
• Demonstrate superior performance and scalability compared to existing systems like Hadoop in real-world benchmarks.

Proposed method

• Design Sector as a storage cloud that manages data both within and across data centers using a distributed, scalable architecture.
• Implement Sphere as a compute cloud that executes UDFs on data stored in Sector, supporting both intra- and inter-data center processing.
• Support MapReduce programming patterns by allowing users to define a Map UDF followed by a Reduce UDF within the Sphere framework.
• Use a decentralized, fault-tolerant design to ensure high availability and data consistency across distributed nodes.
• Leverage commodity hardware and a simplified programming model to reduce operational complexity and improve developer productivity.
• Integrate the system with standard cloud infrastructure to enable seamless deployment and interoperability.

Experimental results

Research questions

• RQ1 How can large-scale data be efficiently stored and processed across multiple, geographically distributed data centers?
• RQ2 To what extent can a unified programming model simplify the development of distributed data processing applications?
• RQ3 Can a system built on UDFs outperform established frameworks like Hadoop in terms of performance and scalability?
• RQ4 What are the performance characteristics of a storage and compute stack designed for cross-data center workloads?
• RQ5 How does the system handle fault tolerance and data consistency in a distributed, multi-data-center environment?

Key findings

• Sector and Sphere achieved approximately twice the performance of Hadoop on the Terasort benchmark, indicating significant speed improvements.
• The system successfully managed data and executed UDFs across multiple data centers, demonstrating scalability beyond single-data-center constraints.
• MapReduce-style workloads were efficiently implemented using a two-phase UDF model (Map then Reduce), validating the extensibility of the programming model.
• The open-source nature of Sector/Sphere enables community adoption and further extension for large-scale data processing.
• The design reduced operational complexity by abstracting low-level distribution details, improving developer productivity.
• The system maintained high availability and fault tolerance through its decentralized architecture, even under distributed workloads.