[Paper Review] Storing and Querying Large-Scale Spatio-Temporal Graphs with High-Throughput Edge Insertions

Mengsu Ding, Muqiao Yang+1 more|arXiv (Cornell University)|Apr 21, 2019

Data Management and Algorithms30 references4 citations

TL;DR

This paper proposes PAST, a distributed framework for storing and querying large-scale spatio-temporal graphs with high-throughput edge insertions. It uses a novel partitioning strategy based on object vertices and spatio-temporal indexing to minimize cross-machine communication, achieving 1–4 orders of magnitude better query performance than state-of-the-art systems like JanusGraph, Greenplum, and ST-Hadoop.

ABSTRACT

Real-world graphs often contain spatio-temporal information and evolve over time. Compared with static graphs, spatio-temporal graphs have very different characteristics, presenting more significant challenges in data volume, data velocity, and query processing. In this paper, we describe three representative applications to understand the features of spatio-temporal graphs. Based on the commonalities of the applications, we define a formal spatio-temporal graph model, where a graph consists of location vertices, object vertices, and event edges. Then we discuss a set of design goals to meet the requirements of the applications: (i) supporting up to 10 billion object vertices, 10 million location vertices, and 100 trillion edges in the graph, (ii) supporting up to 1 trillion new edges that are streamed in daily, and (iii) minimizing cross-machine communication for query processing. We propose and evaluate PAST, a framework for efficient PArtitioning and query processing of Spatio-Temporal graphs. Experimental results show that PAST successfully achieves the above goals. It improves query performance by orders of magnitude compared with state-of-the-art solutions, including JanusGraph, Greenplum, Spark and ST-Hadoop.

Motivation & Objective

• To address the challenges of storing and querying massive-scale spatio-temporal graphs with up to 100 trillion edges and 1 trillion daily edge insertions.
• To minimize cross-machine communication in distributed query processing for spatio-temporal graphs.
• To support real-time, high-velocity data ingestion and complex analytical queries on graphs modeling people, objects, and events over space and time.
• To design a scalable, efficient framework that outperforms existing systems like JanusGraph, Greenplum, and ST-Hadoop in both storage and query workloads.

Proposed method

• Proposes a formal bipartite spatio-temporal graph model with location vertices, object vertices, and event edges representing spatio-temporal interactions.
• Employs a medium-grain hash partitioning scheme for object vertices to balance data distribution and reduce coordination overhead.
• Introduces a spatio-temporal index on edges using time and location to enable efficient pruning of irrelevant data during query processing.
• Designs a coordinator-based load balancing mechanism that dynamically migrates vertex groups based on worker node utilization metrics.
• Optimizes query execution by pushing down selection predicates and minimizing data shuffling across nodes through localized computation.
• Supports integration as an index layer on top of RDBMSs to accelerate hybrid relational and graph workloads.

Experimental results

Research questions

• RQ1 How can a distributed system efficiently store and query spatio-temporal graphs with up to 100 trillion edges and 1 trillion daily edge insertions?
• RQ2 What partitioning strategy minimizes cross-machine communication while maintaining load balance in large-scale graph processing?
• RQ3 How can spatio-temporal indexing reduce the amount of data accessed during complex graph queries?
• RQ4 To what extent can a custom partitioning and indexing framework outperform general-purpose graph and data management systems in real-world workloads?

Key findings

• PAST achieves 1–4 orders of magnitude better query performance than JanusGraph, Greenplum, Spark, and ST-Hadoop across all benchmark queries.
• For large time-range queries (Q3 and Q4), PAST avoids system overload, while Cassandra+Spark and ST-Hadoop+Spark were overwhelmed by excessive data shuffling.
• Greenplum performed well on simple queries due to object-based partitioning, but its performance degraded significantly on complex queries requiring full data scans.
• The spatio-temporal index in PAST reduced data access by up to several orders of magnitude, especially for range-based and similarity-based queries.
• Dynamic load balancing via group migration maintained system efficiency under varying workloads, with coordinator-driven rebalancing keeping node utilization within ±5% of the average.
• Preliminary evaluation shows that PAST can scale to store up to 10 billion object vertices, 10 million location vertices, and 100 trillion edges, meeting the target scale of real-world applications.