[Paper Review] Sorting with GPUs: A Survey

Dmitri I. Arkhipov, Di Wu+2 more|arXiv (Cornell University)|Sep 8, 2017

Algorithms and Data Compression39 references4 citations

TL;DR

This survey provides a comprehensive analysis of GPU-based sorting algorithms, focusing on parallel radix sort, sample sort, and hybrid approaches. It identifies effective use of on-chip memory and efficient GPU primitives—particularly scan and 1-bit scatter—as critical to achieving high performance, with hybrid radix-bitonic sorting emerging as the most efficient approach for large-scale data.

ABSTRACT

Sorting is a fundamental operation in computer science and is a bottleneck in many important fields. Sorting is critical to database applications, online search and indexing,biomedical computing, and many other applications. The explosive growth in computational power and availability of GPU coprocessors has allowed sort operations on GPUs to be done much faster than any equivalently priced CPU. Current trends in GPU computing shows that this explosive growth in GPU capabilities is likely to continue for some time. As such, there is a need to develop algorithms to effectively harness the power of GPUs for crucial applications such as sorting.

Motivation & Objective

• To map and summarize the state of research on GPU-based sorting algorithms as of 2017.
• To identify key performance bottlenecks in GPU sorting, particularly memory access latency and bandwidth limitations.
• To highlight effective algorithmic strategies for optimizing on-chip memory usage and reducing memory contention.
• To evaluate the role of GPU primitives—such as scan and 1-bit scatter—in determining sorting performance.
• To provide a comparative overview of sorting techniques, including radix, sample, and hybrid methods, for researchers seeking high-performance solutions.

Proposed method

• Categorizing existing GPU sorting algorithms into three main families: parallel radix sort, sample sort, and hybrid approaches.
• Analyzing the role of sorting networks (e.g., bitonic and odd-even merge) as subroutines in GPU-based sorting pipelines.
• Evaluating the impact of memory hierarchy optimization, especially maximizing use of shared and register memory over global memory.
• Assessing the influence of GPU primitives—particularly scan and 1-bit scatter—on algorithmic efficiency and parallelism granularity.
• Comparing performance across implementations by measuring communication overhead, memory access patterns, and thread-level load balancing.
• Synthesizing findings from 30+ key papers to identify common design principles and performance-critical optimizations.

Experimental results

Research questions

• RQ1 What are the dominant algorithmic families used in GPU-based sorting, and how do they differ in performance and scalability?
• RQ2 How do memory access patterns and contention affect GPU sorting performance, and what strategies mitigate these issues?
• RQ3 To what extent do GPU-specific primitives (e.g., scan, 1-bit scatter) influence the efficiency of sorting algorithms?
• RQ4 Why does hybrid radix-bitonic sorting outperform other approaches in practice?
• RQ5 How do on-chip memory and register usage correlate with overall sorting throughput on modern GPUs?

Key findings

• Effective use of on-chip memory and registers is the single most important factor in achieving high-performance GPU sorting.
• Hybrid approaches combining radix sort with sorting networks per scalar processor achieve the best performance, outperforming pure comparison-based or radix-only methods.
• Radix sort with bucketization and per-warp sorting networks reduces memory contention and improves load balancing across threads.
• The efficiency of GPU primitives—especially 1-bit scatter and scan—has a significant impact on performance, with some implementations offering better fine-grained parallelism.
• Comparison-based sorts like warp sort and sample sort are competitive with radix-based methods, particularly for 32-bit keys, but are outperformed by hybrid radix approaches for larger data sets.
• Algorithmic improvements that reduce communication and synchronization overhead—especially through register reuse and thread coalescing—yield substantial performance gains over naive kernel mappings.