[Paper Review] A Survey on Temporal Graph Representation Learning and Generative Modeling

Shubham Gupta, Srikanta Bedathur|arXiv (Cornell University)|Aug 25, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This survey unifies temporal graph representation learning and generative modeling, identifying key limitations in existing methods like poor scalability, lack of inductive bias, and high edge overlap. It introduces Tigger, a scalable, inductive generative model for temporal interaction graphs that learns joint structural and temporal distributions from a single graph sample, enabling realistic, diverse graph synthesis without memorizing source edges.

ABSTRACT

Temporal graphs represent the dynamic relationships among entities and occur in many real life application like social networks, e commerce, communication, road networks, biological systems, and many more. They necessitate research beyond the work related to static graphs in terms of their generative modeling and representation learning. In this survey, we comprehensively review the neural time dependent graph representation learning and generative modeling approaches proposed in recent times for handling temporal graphs. Finally, we identify the weaknesses of existing approaches and discuss the research proposal of our recently published paper TIGGER[24].

Motivation & Objective

• To provide a unified overview of temporal graph representation learning and generative modeling, bridging gaps between static graph embedding and dynamic graph generation.
• To identify critical shortcomings in current temporal graph generative models, including poor scalability, non-inductive design, and high edge overlap with source graphs.
• To motivate the need for scalable, inductive generative modeling of temporal interaction graphs that generalize beyond memorizing the input graph.
• To present the problem formulation for Tigger, a novel approach that learns the underlying joint distribution of structural and temporal properties from a single temporal graph.

Proposed method

• Proposes a problem formulation where a generative model learns the hidden joint distribution of structural and temporal properties from a single input temporal interaction graph.
• Introduces Tigger as a scalable, inductive generative model that avoids one-to-one node ID mapping, enabling generation of graphs of different sizes.
• Employs a latent space modeling approach to capture inter-interaction time distributions and evolving graph properties like degree distribution and connected components.
• Uses a differentiable, amortized inference mechanism to model temporal dynamics without relying on static graph conversion, avoiding quadratic memory costs.
• Leverages a one-shot learning paradigm to train on a single graph instance, enabling generalization to unseen graph sizes and temporal horizons.
• Employs a variational inference framework to maximize likelihood of the input graph while ensuring generated graphs are distinct and diverse.

Experimental results

Research questions

• RQ1 How can temporal graph generative models be made scalable to large graphs with high interaction frequency and long time horizons?
• RQ2 What design principles enable inductive generalization in temporal graph generation, avoiding reliance on fixed node ID mappings?
• RQ3 How can generative models learn the joint distribution of structural and temporal properties without memorizing edges from the source graph?
• RQ4 What are the key limitations of existing models like TagGen and DYMOND in terms of scalability, diversity, and inductive bias?
• RQ5 Can a single-graph, one-shot generative model effectively capture the latent dynamics of temporal interaction graphs?

Key findings

• Existing temporal graph generative models like TagGen and DYMOND suffer from poor scalability, with limitations to graphs of fewer than 10,000 nodes and 200 timestamps.
• TagGen exhibits up to 80% edge overlap between generated and source graphs due to explicit edge copying, indicating poor generalization and memorization.
• DYMOND has a computational complexity of O(N³T), making it infeasible for large-scale graphs with high node counts or long time spans.
• Both TagGen and DYMOND are non-inductive, relying on one-to-one node ID mapping, which prevents generalization to graphs of different sizes.
• The survey identifies a critical research gap in scalable, inductive temporal graph generative modeling, motivating the development of Tigger as a solution.
• Tigger addresses these limitations by enabling inductive, scalable generation of diverse temporal graphs from a single input graph without edge memorization.