[Paper Review] Helios: An Efficient Out-of-core GNN Training System on Terabyte-scale Graphs with In-memory Performance

Jie Sun, Mo Sun+7 more|arXiv (Cornell University)|Oct 2, 2023

Advanced Graph Neural NetworksComputer Science3 citations

TL;DR

Helios is a novel out-of-core GNN training system that achieves in-memory training throughput on terabyte-scale graphs by leveraging GPU-initiated asynchronous I/O, a GPU-managed heterogeneous cache across CPU and GPU memory, and a deep GNN-aware pipeline. It achieves up to 6.43× speedup over state-of-the-art GPU-managed baselines and over 182× over CPU-managed systems on terabyte graphs.

ABSTRACT

Training graph neural networks (GNNs) on large-scale graph data holds immense promise for numerous real-world applications but remains a great challenge. Several disk-based GNN systems have been built to train large-scale graphs in a single machine. However, they often fall short in terms of performance, especially when training on terabyte-scale graphs. This is because existing disk-based systems either overly focus on minimizing the number of SSD accesses or do not fully overlap SSD accesses with GNN training, thus resulting in substantial unnecessary overhead on the CPU side and then low GPU utilization. To this end, we propose Helios, a system that can train GNN on terabyte graphs in a single machine while achieving throughput comparable with in-memory systems. To achieve this, we first present a GPU-initiated asynchronous disk IO stack, allowing the GPU to directly access graph data on SSD. This design only requires about 30% GPU cores to reach the almost maximal disk IO throughput and wastes no GPU cores between IO submission and IO completion such that the majority of GPU cores are left for other GNN kernels. Second, we design a GPU-managed heterogeneous cache that extends the cache hierarchy to heterogeneous CPU and GPU memory and thus enhances cache lookup throughput significantly by GPU parallelism. Finally, we build a deep GNN-aware pipeline that seamlessly integrates the computation and communication phases of the entire GNN training process, maximizing the utility of GPU computation cycles. Experimental results demonstrate that Helios can match the training throughput of in-memory GNN systems, even for terabyte-scale graphs. Remarkably, Helios surpasses the state-of-the-art GPU-managed baselines by up to 6.43x and exceeds CPU-managed baselines by over 182x on all terabyte-scale graphs.

Motivation & Objective

• Address the critical performance bottleneck in disk-based GNN training systems that suffer from CPU bottlenecks and low GPU utilization when handling terabyte-scale graphs.
• Overcome the limitations of existing CPU-managed systems, which waste GPU cycles due to serial CPU-GPU data transfer and high CPU overhead in mini-batch preparation.
• Eliminate the throughput ceiling of GPU-managed systems like GIDS, which underutilize disk I/O and exhaust GPU resources due to lack of parallelism and insufficient cache capacity.
• Achieve in-memory-level training throughput for terabyte-scale graphs using a single-machine, out-of-core architecture without requiring in-memory storage.
• Design a system that maximizes GPU utilization by overlapping computation, I/O, and cache management through GPU-native control and intelligent data hierarchy management.

Proposed method

• Introduces a GPU-initiated asynchronous I/O stack that enables direct GPU access to SSD-stored graph data, reducing CPU involvement and allowing 30% of GPU cores to sustain near-maximal disk throughput.
• Employs a GPU-managed heterogeneous cache that spans CPU and GPU memory, leveraging GPU parallelism to accelerate cache lookups and reduce I/O latency.
• Designs a deep GNN-aware pipeline that seamlessly integrates computation and communication phases, minimizing idle GPU cycles and maximizing hardware utilization.
• Optimizes disk I/O by overlapping data loading with GNN training kernels, ensuring continuous GPU workload and eliminating idle time between I/O submission and completion.
• Uses a super-batch and mini-batch generation strategy where GPU-driven sampling and feature extraction are scheduled to minimize redundant data movement and maximize I/O efficiency.
• Leverages GPUDirect technology to enable direct GPU-SSD data transfers, reducing latency and memory copy overhead compared to CPU-mediated data movement.

Experimental results

Research questions

• RQ1 Can a disk-based GNN training system achieve in-memory-level throughput on terabyte-scale graphs by minimizing CPU bottlenecks and maximizing GPU utilization?
• RQ2 To what extent can GPU-initiated asynchronous I/O improve disk I/O throughput and reduce GPU idle time in GNN training workloads?
• RQ3 How effective is a GPU-managed heterogeneous cache across CPU and GPU memory in reducing disk I/O and accelerating data access for large-scale GNNs?
• RQ4 Can a deep, GNN-aware pipeline that integrates computation and communication phases significantly improve GPU utilization and training throughput?
• RQ5 What is the performance gap between CPU-managed and GPU-managed disk-based GNN systems on terabyte-scale graphs, and can a new system close this gap to match in-memory performance?

Key findings

• Helios achieves training throughput comparable to in-memory GNN systems, even on terabyte-scale graphs, by eliminating CPU bottlenecks and maximizing GPU utilization.
• Helios surpasses the state-of-the-art GPU-managed baseline GIDS by up to 6.43× in training throughput on all terabyte-scale graphs.
• Helios exceeds CPU-managed baselines like Ginex and MariusGNN by over 182× in training throughput, demonstrating a dramatic performance advantage.
• The GPU-initiated asynchronous I/O stack achieves near-maximal disk I/O throughput using only about 30% of GPU cores, leaving the majority available for GNN kernels.
• The GPU-managed heterogeneous cache significantly reduces disk I/O by caching frequently accessed data across CPU and GPU memory, mitigating the performance penalty of out-of-core storage.
• Helios achieves over 90% GPU utilization in training workloads, in contrast to the 0.6% utilization observed in CPU-managed systems like Ginex, due to effective overlap of I/O and computation.