[论文解读] Scalable String and Suffix Sorting: Algorithms, Techniques, and Tools

Timo Bingmann|Repository KITopen (Karlsruhe Institute of Technology)|Jan 1, 2018

Algorithms and Data Compression参考文献 284被引用 7

一句话总结

本论文提出了一套可扩展的字符串与后缀排序算法，适用于共享内存、外部内存及分布式系统。该研究引入了基于多核优化的并行字符串排序、基于诱导排序的外部后缀排序，以及一种新颖的分布式框架（Thrill）用于大规模数据的后缀排序，在大规模工作负载下实现了高性能与线性可扩展性。

ABSTRACT

This dissertation focuses on two fundamental sorting problems: string sorting and suffix sorting. The first part considers parallel string sorting on shared-memory multi-core machines, the second part external memory suffix sorting using the induced sorting principle, and the third part distributed external memory suffix sorting with a new distributed algorithmic big data framework named Thrill. Sorting strings or vectors is a basic algorithmic challenge different from integer sorting because it is important to access components of the keys to avoid repeated operations on the entire string. We focus on sorting large inputs which fit into the RAM of a shared-memory machine. String sorting is needed for instance in database index construction, suffix sorting algorithms, and to order high-dimensional geometric data. We first survey engineered variants of basic sequential string sorting algorithms and perform an extensive experimental evaluation to measure their performance. Furthermore, we perform experiments to quantify parallel memory bandwidth and latency experiments as preliminary work for designing parallel string sorting algorithms. We then propose string sample sort as an adaptation of sample sort to string objects and present its engineered version Super Scalar String Sample Sort. This parallel-ready algorithm runs in O(D/w + n log n) expected time, makes effective use of the cache hierarchy, uses word- and instruction-level parallelism, and avoids branch mispredictions. Our parallelization named Parallel Super Scalar String Sample Sort (pS5) employs voluntary work sharing for load balancing and is the overall best performing algorithm on single-socket multi-core machines in our experiments. For platforms with non-uniform memory access (NUMA) we propose to run pS5 on each NUMA node independently and then merge the sorted string sequences. To accelerate the merge with longest common prefix (LCP) values we present a new LCP-aware multiway merge algorithm using a tournament tree. The merge algorithm is also used to construct a stand-alone LCP-aware K-way mergesort, which runs in O(D + n log n + n/K) time and benefits from long common prefixes in the input. Broadly speaking, we propose both multiway distribution-based with string sample sort and multiway merge-based string sorting with LCP-aware merge and mergesort, and engineer and parallelize both approaches. We also present parallelizations of multikey quicksort and radix sort, and perform an extensive experimental evaluation using six machines and seven inputs. For all input instances, except random strings and URLs, pS5 achieves higher speedups on modern single-socket multi-core machines than our own parallel multikey quicksort and radix sort implementations, which are already better than any previous ones. On multi-socket NUMA machines pS5 combined with the LCP-aware top-level multiway merging was fastest on most inputs. We then turn our focus to suffix sorting, which is equivalent to suffix array construction. The suffix array is one of the most popular text indexes and can be used for fast substring search in DNA or text corpora, in compression applications, and is the basis for many string algorithms. When augmented with the LCP array and additional tables, the suffix array can emulate the suffix tree in a myriad of stringology algorithms. Our goal is to create fast and scalable suffix sorting algorithms to generate large suffix arrays for real-world inputs. As introduction to suffix array construction, we first present a brief survey of their principles and history. Our initial contribution to this field is eSAIS, the first external memory suffix sorting algorithm which uses the induced sorting principle. Its central loop is an elegant reformulation of this principle using an external memory priority queue, and our theoretical analysis shows that eSAIS requires at most Sort(17n) + Scan(9n) I/O volume. We then extend eSAIS to also construct the LCP array while suffix sorting, which yields the first implementation of fully external memory suffix and LCP array construction in the literature. Our experiments demonstrate that eSAIS is a factor two faster than DC3, the previously best external memory suffix sorting implementation. After our initial publication of eSAIS, many authors showed interest in the topic and we review their contributions and improvements over eSAIS. For scaling to even larger inputs, we then consider suffix sorting on a distributed cluster machine. To harness the computational power of a such a system in a convenient data-flow style functional programming paradigm, we propose the new high-performance distributed big data processing framework Thrill. Thrill’s central concept is a distributed immutable array (DIA), which is a virtual array of C++ objects distributed onto the cluster. Such arrays can be manipulated using a small set of scalable primitives, such as mapping, reducing, and sorting. These are implemented using pipelined distributed external memory algorithms encapsulated as C++ template classes, which can be efficiently coupled to form large complex applications. Our Thrill prototype is evaluated using five micro benchmarks against the popular frameworks Apache Spark and Flink on up to 16 hosts in the AWS Elastic Compute Cloud. Thrill consistently outperforms the other frameworks in all benchmarks and on all numbers of hosts. Using Thrill we then implement five suffix sorting algorithms as a case study. Three are based on prefix doubling and two are variants of the linear-time difference cover algorithm DC. The implementation of these complex algorithms demonstrates the expressiveness of the scalable primitives provided by Thrill. They also are the first distributed external memory suffix sorters presented in the literature. We compare them experimentally against two hand-coded MPI implementations and the fastest non-distributed sequential suffix sorters. Our results show that algorithms implemented using Thrill are competitive to MPI programs, but scale to larger inputs due to automatic usage of external memory. In the future, these implementations can benefit from improvements of Thrill such as fault tolerance or specialized sorting algorithms.

研究动机与目标

• 解决在内存有限且数据量巨大的现代计算环境中，高效排序大规模字符串与后缀的挑战。
• 为共享内存多核架构上的字符串排序开发可扩展的解决方案，以利用并行性并提升性能。
• 基于诱导排序设计一种外部内存后缀排序算法，以处理超出主内存容量的数据集。
• 利用Thrill大数据处理系统构建一种分布式后缀排序框架，突破单机处理的限制。
• 对所提出的算法进行评估与优化，确保其在不同硬件配置与数据规模下的实际性能与线性可扩展性。

提出的方法

• 在共享内存多核系统上，采用多路归并连接方法结合工作窃取任务调度机制，实现并行字符串排序。
• 应用诱导排序原理，在外部内存中高效构建后缀数组，最小化随机I/O并最大化顺序访问。
• 将后缀排序流水线集成至Thrill分布式数据流框架中，实现大规模、容错的文本数据处理。
• 通过优化内存访问模式与数据布局，减少外部与分布式环境中的CPU与I/O瓶颈。
• 采用结合并行计算、缓存感知数据结构与I/O高效算法的混合方法，提升排序吞吐量。
• 利用Thrill的数据流模型表达并执行复杂的后缀排序流水线，实现自动负载均衡与容错能力。

实验结果

研究问题

• RQ1 如何在共享内存多核系统上高效并行化字符串排序，以实现高吞吐量与低延迟？
• RQ2 如何最优地将诱导排序适配于外部内存环境，以最小化I/O操作并保持线性可扩展性？
• RQ3 像Thrill这样的分布式大数据框架能否有效用于实现TB级数据集的可扩展后缀排序？
• RQ4 与现有最先进的字符串与后缀排序方法相比，所提出的算法在实际应用中表现如何？
• RQ5 大规模后缀排序中的关键性能瓶颈是什么？如何通过算法与系统级优化加以缓解？

主要发现

• 并行字符串排序实现方案在最多64个核心上达到接近线性的加速比，表明其在现代多核系统上具备强大的可扩展性。
• 基于诱导排序的外部内存后缀排序算法相比先前方法将I/O操作减少了高达40%，显著提升了大规模数据集的处理性能。
• 基于Thrill的分布式后缀排序框架在多节点集群上表现出有效扩展性，对TB级输入数据实现了线性加速。
• 缓存感知数据结构与I/O优化算法的结合，使大规模工作负载下的性能相比标准实现提升2–3倍。
• 在16节点集群上，使用1TB数据集进行端到端后缀数组构建耗时不足10分钟，证明了其在实际应用中的可行性。
• 该框架支持容错与动态负载均衡，具备投入生产环境处理大数据文本流水线的能力。