[Paper Review] Efficient Resource Sharing Through GPU Virtualization on Accelerated High Performance Computing Systems

Teng Li, Vikram K. Narayana+1 more|arXiv (Cornell University)|Nov 24, 2015

Parallel Computing and Optimization Techniques12 references3 citations

TL;DR

This paper proposes a GPU virtualization framework that enables efficient sharing of GPUs among multiple microprocessors in heterogeneous HPC systems under the SPMD model. By managing concurrent kernel execution and overlapping I/O and computation, the approach achieves up to 7.4x speedup with low virtualization overhead, enabling full utilization of underused CPU cores and improving GPU resource efficiency in imbalanced CPU/GPU clusters.

ABSTRACT

The High Performance Computing (HPC) field is witnessing a widespread adoption of Graphics Processing Units (GPUs) as co-processors for conventional homogeneous clusters. The adoption of prevalent Single- Program Multiple-Data (SPMD) programming paradigm for GPU-based parallel processing brings in the challenge of resource underutilization, with the asymmetrical processor/co-processor distribution. In other words, under SPMD, balanced CPU/GPU distribution is required to ensure full resource utilization. In this paper, we propose a GPU resource virtualization approach to allow underutilized microprocessors to effi- ciently share the GPUs. We propose an efficient GPU sharing scenario achieved through GPU virtualization and analyze the performance potentials through execution models. We further present the implementation details of the virtualization infrastructure, followed by the experimental analyses. The results demonstrate considerable performance gains with GPU virtualization. Furthermore, the proposed solution enables full utilization of asymmetrical resources, through efficient GPU sharing among microprocessors, while incurring low overhead due to the added virtualization layer.

Motivation & Objective

• Address the underutilization of microprocessors in GPU-accelerated HPC systems due to imbalanced CPU/GPU ratios.
• Overcome the SPMD programming model's limitation of requiring a one-to-one CPU-GPU mapping for full resource utilization.
• Enable efficient GPU sharing among multiple microprocessors without sacrificing performance or incurring high virtualization overhead.
• Develop a runtime virtualization layer that exposes a virtual GPU (vGPU) to each microprocessor, abstracting physical GPU resource contention.
• Demonstrate that virtualization can achieve significant performance gains through concurrency, overlapping, and reduced context-switching overhead.

Proposed method

• Design a user-space runtime virtualization layer that manages GPU resource allocation across multiple microprocessors.
• Expose a virtual GPU (vGPU) interface to each process, simulating dedicated GPU access while sharing a single physical GPU.
• Enable concurrent execution of multiple GPU kernels on a single GPU by exploiting the Fermi architecture’s support for up to 16 concurrent kernels.
• Optimize for overlapping of I/O and kernel computation by scheduling GPU operations to hide latency.
• Minimize context-switching and initialization overhead by maintaining persistent GPU state and resource pools across processes.
• Develop an analytical execution model to predict performance gains and guide virtualization design based on application profiles.

Experimental results

Research questions

• RQ1 Can GPU virtualization effectively enable multiple microprocessors to share a single GPU under the SPMD execution model?
• RQ2 What performance gains can be achieved through virtualization, and how do they vary across different application types (I/O-intensive, compute-intensive, mixed)?
• RQ3 To what extent can I/O and kernel computation be overlapped in a virtualized GPU environment?
• RQ4 How does the virtualization overhead compare to direct GPU access, and can it be kept low across diverse workloads?
• RQ5 Can the proposed virtualization framework achieve high GPU utilization in systems with imbalanced CPU/GPU ratios?

Key findings

• The proposed GPU virtualization framework achieved speedups ranging from 1.4x to 7.4x across seven benchmarks under 8-process parallelism.
• Compute-intensive benchmarks like MG and CG achieved the highest performance gains (up to 7.4x) due to effective overlapping of I/O and kernel execution.
• I/O-intensive benchmarks like Black Scholes and VecAdd achieved moderate gains (1.4x–2.5x) primarily through I/O and kernel overlapping.
• Large compute-intensive kernels like Electrostatic (ES) achieved limited gains (around 1.5x) due to minimal overlapping potential and high GPU occupancy.
• The virtualization layer incurred low overhead, with experimental results closely matching the analytical performance model.
• The framework successfully enabled full utilization of underused CPU cores by efficiently sharing GPUs, even in systems with high CPU/GPU ratios.