[论文解读] High-Quality Hypergraph Partitioning

Sebastian Schlag|arXiv (Cornell University)|Jan 1, 2020

VLSI and FPGA Design Techniques参考文献 113被引用 16

一句话总结

本文提出 KaHyPar，一个开源的超图划分框架，引入了具有先进粗化、细化和记忆启发式算法的 n 级多级范式，在各种超图上，于切割网和连通性度量上均实现了卓越的解质量与速度，优于 hMETIS、PaToH、Mondriaan、Zoltan-AlgD、HYPE 和 KaFFPa。

ABSTRACT

This dissertation focuses on computing high-quality solutions for the NP-hard $ extit{balanced hypergraph partitioning problem}$ : Given a hypergraph and an integer $k$, partition its vertex set into $k$ disjoint blocks of bounded size, while minimizing an objective function over the hyperedges. Here, we consider the two most commonly used objectives: the $ extit{cut-net}$ metric and the $ extit{connectivity}$ metric. Since the problem is computationally intractable, heuristics are used in practice -- the most prominent being the three-phase multi-level paradigm: During coarsening, the hypergraph is successively contracted to obtain a hierarchy of smaller instances. After applying an initial partitioning algorithm to the smallest hypergraph, contraction is undone and, at each level, refinement algorithms try to improve the current solution. With this work, we give a brief overview of the field and present several algorithmic improvements to the multi-level paradigm. Instead of using a logarithmic number of levels like traditional algorithms, we present two coarsening algorithms that create a hierarchy of (nearly) $n$ levels, where $n$ is the number of vertices. This makes consecutive levels as similar as possible and provides many opportunities for refinement algorithms to improve the partition. This approach is made feasible in practice by tailoring all algorithms and data structures to the $n$-level paradigm, and developing lazy-evaluation techniques, caching mechanisms and early stopping criteria to speed up the partitioning process. Furthermore, we propose a sparsification algorithm based on locality-sensitive hashing that improves the running time for hypergraphs with large hyperedges, and show that incorporating global information about the community structure into the coarsening process improves quality. Moreover, we present a portfolio-based initial partitioning approach, and propose three refinement algorithms. Two are based on the Fiduccia-Mattheyses (FM) heuristic, but perform a highly localized search at each level. While one is designed for two-way partitioning, the other is the first FM-style algorithm that can be efficiently employed in the multi-level setting to directly improve $k$-way partitions. The third algorithm uses max-flow computations on pairs of blocks to refine $k$-way partitions. Finally, we present the first memetic multi-level hypergraph partitioning algorithm for an extensive exploration of the global solution space. All contributions are made available through our open-source framework KaHyPar. In a comprehensive experimental study, we compare KaHyPar with hMETIS, PaToH, Mondriaan, Zoltan-AlgD, and HYPE on a wide range of hypergraphs from several application areas. Our results indicate that KaHyPar, already without the memetic component, computes better solutions than all competing algorithms for both the cut-net and the connectivity metric, while being faster than Zoltan-AlgD and equally fast as hMETIS. Moreover, KaHyPar compares favorably with the current best graph partitioning system KaFFPa -- both in terms of solution quality and running time.

研究动机与目标

• 为解决 NP 难的平衡超图划分问题，提升解质量与效率。
• 通过引入非对数级的 n 级粗化层次结构，克服传统多级划分方法的局限性。
• 通过整合全局社区结构和局部敏感哈希技术，提升大超边的划分质量。
• 开发针对 k 路划分的高效细化算法，包括 FM 风格和最大流基础方法。
• 通过一种新颖的记忆启发式多级算法，实现解空间的全局探索。

提出的方法

• 设计一种近似 n 级粗化层次结构，使用两种新型粗化算法，以最大化连续层级之间的相似性。
• 在 n 级设置中实现惰性求值、缓存和早期停止机制，以优化性能。
• 引入基于局部敏感哈希的稀疏化技术，加速处理具有大超边的超图。
• 在粗化过程中融入全局社区结构，以指导更优的初始划分决策。
• 提出基于组合的初始划分策略，以多样化初始解。
• 开发三种细化算法：两种 FM 风格启发式（一种用于两路划分，一种用于 k 路划分）以及一种基于最大流的 k 路细化方法。

实验结果

研究问题

• RQ1 与对数级方法相比，n 级多级框架是否能显著提升超图划分的质量？
• RQ2 在粗化过程中引入全局社区结构对解质量有何影响？
• RQ3 基于局部敏感哈希的稀疏化技术在具有大超边的超图上，能在多大程度上减少运行时间？
• RQ4 FM 风格细化能否有效适配于多级设置中直接改进 k 路划分？
• RQ5 记忆启发式多级算法是否能实现更优的全局搜索并提升超图划分的解质量？

主要发现

• KaHyPar 在切割网和连通性度量上均优于 hMETIS、PaToH、Mondriaan、Zoltan-AlgD 和 HYPE。
• 即使不启用记忆启发式组件，KaHyPar 在所有基准超图上仍优于所有对比算法的解质量。
• KaHyPar 的运行速度优于 Zoltan-AlgD，且与 hMETIS 速度相当，展现出高效率。
• n 级粗化方法由于层级间更精细的相似性，使得细化过程更加有效。
• 所提出的 FM 风格 k 路细化算法是首个在多级划分中实现高效 FM 基方法直接改进 k 路划分的算法。
• 记忆启发式多级算法实现了广泛的全局探索，进一步提升了标准细化方法的解质量。