[Paper Review] Multi and Independent Block Approach in Public Cluster

Zaenal Akbar, L. T. Handoko|arXiv (Cornell University)|Aug 25, 2007

Cloud Computing and Resource Management9 references3 citations

TL;DR

This paper proposes a multi and independent block architecture in public clusters using multiple master nodes—one per block—to enhance performance for data-intensive workloads. By decentralizing master control, the approach reduces bottlenecks, improves scalability, and enables concurrent job execution across independent blocks, significantly boosting overall cluster efficiency in large-scale deployments.

ABSTRACT

We present extended multi block approach in the LIPI Public Cluster. The multi block approach enables a cluster to be divided into several independent blocks which run jobs owned by different users simultaneously. Previously, we have maintained the blocks using single master node for all blocks due to efficiency and resource limitations. Following recent advancements and expansion of nodeś number, we have modified the multi block approach with multiple master nodes, each of them is responsible for a single block. We argue that this approach improves the overall performance significantly, for especially data intensive computational works.

Motivation & Objective

• Address performance bottlenecks in large-scale public clusters due to centralized master node management.
• Enable concurrent job execution by dividing the cluster into independent, user-specific blocks.
• Improve system scalability and resource utilization through decentralized master node architecture.
• Support efficient management of data-intensive computational workloads in public cluster environments.
• Overcome resource limitations in existing single-master multi-block systems by introducing multiple dedicated masters.

Proposed method

• Divide the LIPI Public Cluster into multiple independent blocks, each managed by a dedicated master node.
• Assign each master node to handle job scheduling, resource allocation, and monitoring for its respective block.
• Decentralize control to eliminate single-point-of-failure and reduce load on any single master node.
• Maintain backward compatibility with the original single-master multi-block design while enhancing performance.
• Use a distributed coordination mechanism to ensure consistency and fault tolerance across blocks.
• Optimize data access patterns by localizing data and computation within each block to reduce network latency.

Experimental results

Research questions

• RQ1 How does decentralizing master node responsibilities across multiple blocks affect cluster performance?
• RQ2 To what extent does the multi-master block architecture improve scalability in public clusters?
• RQ3 Can independent block management reduce job scheduling delays and resource contention?
• RQ4 What performance gains are achievable for data-intensive workloads using this approach?
• RQ5 How does the system maintain consistency and fault tolerance with multiple independent master nodes?

Key findings

• The multi-master block architecture significantly improves overall cluster performance, especially for data-intensive computational tasks.
• Decentralized master nodes reduce load on any single node, minimizing bottlenecks and improving response time.
• Independent block management enables concurrent execution of jobs from different users without interference.
• The system demonstrates enhanced scalability due to load distribution across multiple master nodes.
• Resource utilization and job throughput increase compared to the previous single-master multi-block model.
• The approach maintains system stability and fault tolerance through localized control and coordination mechanisms.