[Paper Review] High-Performance Massive Subgraph Counting using Pipelined Adaptive-Group Communication

Langshi Chen, Bo Peng+5 more|arXiv (Cornell University)|Apr 25, 2018

Graph Theory and Algorithms24 references3 citations

TL;DR

This paper proposes a high-performance, pipelined adaptive-group communication framework for massive subgraph counting using color-coding, enabling efficient parallelization across distributed systems. By integrating pipelined communication, fine-grained task partitioning, and memory-optimized intermediate data handling, the approach achieves 5x speedup and 2x lower peak memory usage compared to state-of-the-art MPI-Fascia on large templates (12–15 vertices) and billion-edge graphs.

ABSTRACT

Subgraph counting aims to count the number of occurrences of a subgraph T (aka as a template) in a given graph G. The basic problem has found applications in diverse domains. The problem is known to be computationally challenging - the complexity grows both as a function of T and G. Recent applications have motivated solving such problems on massive networks with billions of vertices. In this chapter, we study the subgraph counting problem from a parallel computing perspective. We discuss efficient parallel algorithms for approximately resolving subgraph counting problems by using the color-coding technique. We then present several system-level strategies to substantially improve the overall performance of the algorithm in massive subgraph counting problems. We propose: 1) a novel pipelined Adaptive-Group communication pattern to improve inter-node scalability, 2) a fine-grained pipeline design to effectively reduce the memory space of intermediate results, 3) partitioning neighbor lists of subgraph vertices to achieve better thread concurrency and workload balance. Experimentation on an Intel Xeon E5 cluster shows that our implementation achieves 5x speedup of performance compared to the state-of-the-art work while reduces the peak memory utilization by a factor of 2 on large templates of 12 to 15 vertices and input graphs of 2 to 5 billions of edges.

Motivation & Objective

• Address the scalability limitations of existing parallel subgraph counting tools on large-scale graphs with complex, high-vertex templates.
• Overcome communication bottlenecks, load imbalance, and high memory usage in irregular, sparse graph workloads during subgraph counting.
• Enable efficient parallel execution of color-coding-based subgraph counting for tree templates up to 15 vertices on billion-edge graphs.
• Reduce peak memory utilization and improve inter-node communication efficiency in distributed subgraph counting workloads.

Proposed method

• Introduces a novel pipelined adaptive-group communication pattern that interleaves computation and communication to reduce idle time and improve inter-node scalability.
• Employs fine-grained task partitioning of neighbor lists to balance thread-level workloads within each node and mitigate load imbalance.
• Applies intermediate data partitioning with overlapping pipeline stages to reduce peak memory usage by slicing and overlapping data processing.
• Uses the Harp-DAAL framework to dynamically switch communication modes based on template size, adapting to varying computation-to-communication ratios.
• Leverages the color-coding technique for approximate subgraph counting, focusing on tree templates (treelets), with optimized data structures and parallel execution models.
• Implements a data-driven pipeline design that allows continuous data flow between computation and communication phases, minimizing idle cycles.

Experimental results

Research questions

• RQ1 How can communication overhead be minimized in distributed subgraph counting for irregular, sparse graphs with high computation-to-communication ratios?
• RQ2 What techniques can effectively balance workloads across threads within a node when processing sparse, irregular graph data?
• RQ3 How can peak memory utilization be reduced during massive subgraph counting with large templates (12–15 vertices) and billion-edge graphs?
• RQ4 To what extent can pipelined, adaptive communication patterns improve scalability and performance for subgraph counting beyond the limits of existing MPI-based tools?
• RQ5 Can a hybrid communication and computation pipeline maintain high efficiency and low memory usage while scaling to templates of 15 vertices on real-world massive graphs?

Key findings

• The proposed Harp-DAAL AdaptiveLB implementation achieves up to 5x speedup over the state-of-the-art MPI-Fascia on large templates (u12-2) in the Twitter graph (2B edges).
• For templates of size 12–15 vertices, peak memory utilization is reduced by a factor of 2 compared to MPI-Fascia, enabling execution on 120 GB memory nodes.
• On the Friendster graph (0.66B vertices, 5B edges), the system successfully scales to 12-vertex templates, which MPI-Fascia cannot handle due to memory limits.
• Communication overhead is reduced from 80% (MPI-Fascia at u10-2) to around 40% (Harp-DAAL) for large templates, thanks to adaptive-group communication and pipelining.
• Strong scaling from 8 to 16 nodes shows better speedup with Harp-DAAL AdaptiveLB than MPI-Fascia, especially for larger templates.
• For highly skewed datasets, performance improvements reach up to 9x in execution time, demonstrating robustness to data irregularity.