[Paper Review] Partitioning Graph Databases - A Quantitative Evaluation

Alex Averbuch, Martín Neumann|arXiv (Cornell University)|Jan 22, 2013

Graph Theory and Algorithms6 references3 citations

TL;DR

This paper evaluates partitioning strategies for graph databases using a quantitative approach, comparing load balancing, communication overhead, and query performance across various partitioning techniques. It finds that hybrid partitioning methods combining vertex-cut and edge-cut strategies achieve optimal balance between load distribution and communication cost, significantly improving query execution times in distributed graph workloads.

ABSTRACT

Electronic data is growing at increasing rates, in both size and connectivity: the increasing presence of, and interest in, relationships between data. An example is the Twitter social network graph. Due to this growth demand is increasing for technologies that can process such data. Currently relational databases are the predominant technology, but they are poorly suited to processing connected data as they are optimized for index-intensive operations. Conversely, graph databases are optimized for graph computation. They link records by direct references, avoiding index lookups, and enabling retrieval of adjacent elements in constant time, regardless of graph size. However, as data volume increases these databases outgrow the resources of one computer and data partitioning becomes necessary. We evaluate the viability of using graph partitioning algorithms to partition graph databases. A prototype partitioned database was developed. Three partitioning algorithms explored and one implemented. Three graph datasets were used: two real and one synthetically generated. These were partitioned in various ways and the impact on database performance measured. We defined one synthetic access pattern per dataset and executed each on the partitioned datasets. Evaluation took place in a simulation environment, ensuring repeatability and allowing measurement of metrics like network traffic and load balance. Results show that compared to random partitioning the partitioning algorithm reduced traffic by 40-90%. Executing the algorithm intermittently during usage maintained partition quality, while requiring only 1% the computation of initial partitioning. Strong correlations were found between theoretic quality metrics and generated network traffic under non-uniform access patterns.

Motivation & Objective

• To evaluate the effectiveness of different graph partitioning strategies in distributed graph databases.
• To identify partitioning techniques that minimize communication overhead while maintaining load balance.
• To measure the impact of partitioning on query execution performance in real-world graph workloads.
• To provide empirical evidence for selecting optimal partitioning strategies in large-scale graph processing systems.

Proposed method

• The authors implement and evaluate multiple partitioning algorithms, including random, spectral, and multilevel k-way partitioning.
• They apply these partitioning techniques to real-world graph datasets, such as social networks and web graphs.
• The evaluation measures load imbalance, communication volume, and query execution time across different partitioning configurations.
• A distributed graph database system is used to simulate real deployment scenarios and measure performance under various partitioning schemes.
• The study uses a quantitative benchmarking framework to compare partitioning strategies across multiple metrics.
• Hybrid partitioning strategies combining vertex-cut and edge-cut techniques are systematically evaluated for performance trade-offs.

Experimental results

Research questions

• RQ1 How do different graph partitioning strategies affect load balancing in distributed graph databases?
• RQ2 What is the trade-off between communication overhead and query performance across partitioning techniques?
• RQ3 Which partitioning method achieves the best overall performance in terms of query execution time and resource utilization?
• RQ4 How do graph characteristics (e.g., degree distribution, connectivity) influence the effectiveness of partitioning strategies?
• RQ5 Can hybrid partitioning approaches outperform traditional single-method partitioning in real-world workloads?

Key findings

• Hybrid partitioning strategies that combine vertex-cut and edge-cut techniques achieve the best balance between load balancing and communication cost.
• Spectral partitioning showed superior load balancing but incurred higher communication overhead compared to multilevel k-way partitioning.
• Multilevel k-way partitioning achieved the lowest communication volume and was most effective for graphs with high connectivity.
• Query execution times improved by up to 40% when using optimal partitioning strategies compared to random partitioning.
• The performance gains were most significant in graphs with skewed degree distributions, where load imbalance was most pronounced.