[Paper Review] High-throughput Execution of Hierarchical Analysis Pipelines on Hybrid Cluster Platforms

George Teodoro, Tony Pan+4 more|arXiv (Cornell University)|Sep 14, 2012

Distributed and Parallel Computing Systems34 references3 citations

TL;DR

This paper presents a hybrid CPU-GPU runtime middleware that enables high-throughput execution of hierarchical data analysis pipelines on cluster platforms by combining bag-of-tasks and coarse-grain dataflow execution. It achieves 150 image tiles per second on 100 nodes using performance-aware scheduling, data locality optimization, and asynchronous data transfers, significantly outperforming monolithic or coarser-grain approaches.

ABSTRACT

We propose, implement, and experimentally evaluate a runtime middleware to support high-throughput execution on hybrid cluster machines of large-scale analysis applications. A hybrid cluster machine consists of computation nodes which have multiple CPUs and general purpose graphics processing units (GPUs). Our work targets scientific analysis applications in which datasets are processed in application-specific data chunks, and the processing of a data chunk is expressed as a hierarchical pipeline of operations. The proposed middleware system combines a bag-of-tasks style execution with coarse-grain dataflow execution. Data chunks and associated data processing pipelines are scheduled across cluster nodes using a demand driven approach, while within a node operations in a given pipeline instance are scheduled across CPUs and GPUs. The runtime system implements several optimizations, including performance aware task scheduling, architecture aware process placement, data locality conscious task assignment, and data prefetching and asynchronous data copy, to maximize utilization of the aggregate computing power of CPUs and GPUs and minimize data copy overheads. The application and performance benefits of the runtime middleware are demonstrated using an image analysis application, which is employed in a brain cancer study, on a state-of-the-art hybrid cluster in which each node has two 6-core CPUs and three GPUs. Our results show that implementing and scheduling application data processing as a set of fine-grain operations provide more opportunities for runtime optimizations and attain better performance than a coarser-grain, monolithic implementation. The proposed runtime system can achieve high-throughput processing of large datasets - we were able to process an image dataset consisting of 36,848 4Kx4K-pixel image tiles at about 150 tiles/second rate on 100 nodes.

Motivation & Objective

• To address the challenge of efficiently utilizing hybrid CPU-GPU cluster systems for large-scale scientific data analysis.
• To improve performance and resource utilization in data-intensive applications by enabling fine-grained, hierarchical pipeline execution across heterogeneous hardware.
• To minimize data movement and maximize GPU utilization through intelligent scheduling and data management techniques.
• To demonstrate the feasibility of high-throughput processing of terabyte-scale biomedical image datasets using modern hybrid cluster architectures.
• To extend support to MapReduce-style processing for future stages of complex analysis pipelines.

Proposed method

• The middleware combines bag-of-tasks and coarse-grain dataflow execution models to schedule data chunks and pipelines across hybrid cluster nodes.
• It employs a demand-driven scheduling strategy to dynamically assign tasks based on resource availability and performance characteristics.
• Performance-aware task scheduling selects optimal CPU or GPU function variants based on predicted speedups, reducing runtime overhead.
• Architecture-aware process placement ensures efficient mapping of operations to CPUs and GPUs within each node, considering memory and compute capacity.
• Data locality-conscious task assignment minimizes data transfer costs by co-locating computation with data.
• Asynchronous data copying and prefetching reduce idle time and improve overlap between computation and data movement.

Experimental results

Research questions

• RQ1 Can a hybrid runtime middleware achieve high-throughput execution of hierarchical analysis pipelines on CPU-GPU cluster platforms?
• RQ2 How does fine-grained pipeline decomposition compare to monolithic implementation in terms of performance and scheduling flexibility?
• RQ3 To what extent can performance-aware scheduling and data locality optimizations improve GPU utilization and reduce I/O bottlenecks?
• RQ4 What is the scalability of the middleware across 100 hybrid cluster nodes for large-scale image analysis workloads?
• RQ5 How effective are asynchronous data transfers and prefetching in hiding data movement latency?

Key findings

• The middleware achieved a processing rate of 150 image tiles per second on 100 nodes when processing 36,848 4K×4K image tiles from a glioblastoma study.
• The system achieved 77% efficiency on 100 nodes, with a 1.3× speedup over the FCFS baseline, primarily limited by I/O contention under high parallelism.
• Performance-aware scheduling with function variants reduced execution time by minimizing suboptimal CPU-GPU task assignments, even with imperfect speedup predictions.
• Fine-grain pipeline decomposition enabled better scheduling and optimization opportunities, resulting in significantly better performance than monolithic implementations.
• When I/O overhead was excluded, theoretical efficiency reached 93%, indicating that the core scheduling and execution optimizations are highly effective.
• The system demonstrated feasibility for rapid exploration of scientific questions and sensitivity studies by enabling sub-four-minute processing of a terabyte-scale dataset.