[Paper Review] Portal: A Query Language for Evolving Graphs

Vera Zaychik Moffitt, Julia Stoyanovich|arXiv (Cornell University)|Feb 2, 2016

Graph Theory and Algorithms27 references3 citations

TL;DR

Portal is a declarative query language for scalable analysis of evolving graphs, implemented on Apache Spark to support temporal operations like selection, join, aggregation, and advanced analytics. It achieves efficient performance through optimized physical representations and partitioning strategies that balance structural and temporal locality.

ABSTRACT

Graphs are used to represent a plethora of phenomena, from the Web and social networks, to biological pathways, to semantic knowledge bases. Arguably the most interesting and important questions one can ask about graphs have to do with their evolution. Which Web pages are showing an increasing popularity trend? How does influence propagate in social networks? How does knowledge evolve? Much research and engineering effort today goes into developing sophisticated graph analytics and their efficient implementations, both stand-alone and in scope of data processing platforms. Yet, systematic support for scalable querying and analytics over evolving graphs still lacks. In this paper we present Portal, a declarative language that supports efficient querying and exploratory analysis of evolving graphs, and an implementation of Portal in scope of Apache Spark, an open-source distributed data processing framework. Our language supports a variety of operations including temporal selection, join and aggregation, and a rich class of analytics. We develop multiple physical representations and partitioning strategies and study the trade-offs between structural and temporal locality. We provide an extensive experimental evaluation of our system, demonstrating that careful engineering can lead to good performance.

Motivation & Objective

• Address the lack of systematic, scalable support for querying and analyzing evolving graphs in modern data processing platforms.
• Enable efficient exploratory analysis of dynamic graph phenomena such as popularity trends, influence propagation, and knowledge evolution.
• Design a declarative language that supports temporal operations and complex analytics over evolving graphs.
• Implement Portal within Apache Spark to ensure compatibility with large-scale distributed data processing pipelines.
• Optimize performance through tailored physical representations and partitioning strategies that exploit both structural and temporal locality.

Proposed method

• Design a declarative query language, Portal, that supports temporal selection, temporal joins, aggregations, and a rich set of graph analytics over evolving graphs.
• Implement Portal as a library within Apache Spark to leverage its distributed execution engine and scalable data processing capabilities.
• Develop multiple physical representations of evolving graphs to balance storage efficiency and query performance.
• Introduce partitioning strategies that optimize for both structural locality (graph topology) and temporal locality (time-evolving data).
• Evaluate trade-offs between different physical models and partitioning schemes to guide system configuration for varying workloads.
• Use Spark’s execution model to push down predicates and optimize execution plans for temporal and structural operations.

Experimental results

Research questions

• RQ1 How can a declarative query language be designed to efficiently express and process queries over evolving graphs?
• RQ2 What physical representations and partitioning strategies best balance structural and temporal locality in distributed graph processing?
• RQ3 To what extent can Portal achieve scalable performance for complex temporal and analytical queries on evolving graphs?
• RQ4 How do different design choices in physical modeling and partitioning affect query execution time and resource utilization?

Key findings

• Portal enables efficient and scalable querying of evolving graphs through a declarative language integrated with Apache Spark.
• The choice of physical representation and partitioning strategy significantly impacts performance, with trade-offs between structural and temporal locality.
• Optimized partitioning strategies improve query execution time by reducing data shuffling and improving cache locality.
• The system demonstrates good performance across diverse workloads, including temporal selection, joins, and complex analytics on evolving graphs.
• Careful engineering of the physical layer leads to measurable performance gains, validating the importance of low-level optimizations in distributed graph systems.