[Paper Review] Representation Learning for Dynamic Graphs: A Survey

Seyed Mehran Kazemi, Rishab Goel+5 more|arXiv (Cornell University)|May 27, 2019

Advanced Graph Neural NetworksComputer Science250 references69 citations

TL;DR

A comprehensive survey of neural representation learning techniques for dynamic graphs, organized around encoder–decoder frameworks, covering discrete/continuous-time dynamics, KG/HIN settings, and applications. It discusses models, expressivity, datasets, tasks, and open research directions.

ABSTRACT

Graphs arise naturally in many real-world applications including social networks, recommender systems, ontologies, biology, and computational finance. Traditionally, machine learning models for graphs have been mostly designed for static graphs. However, many applications involve evolving graphs. This introduces important challenges for learning and inference since nodes, attributes, and edges change over time. In this survey, we review the recent advances in representation learning for dynamic graphs, including dynamic knowledge graphs. We describe existing models from an encoder-decoder perspective, categorize these encoders and decoders based on the techniques they employ, and analyze the approaches in each category. We also review several prominent applications and widely used datasets and highlight directions for future research.

Motivation & Objective

• Explain the challenges and motivation for representation learning on dynamic graphs.
• Provide an encoder–decoder taxonomy for dynamic-graph models across time regimes (continuous and discrete).
• Survey encoders and decoders, their expressivity, and training paradigms for dynamic graphs.
• Highlight common tasks, datasets, applications, and open problems to guide future research.

Proposed method

• Define dynamic graph formalisms (continuous-time and discrete-time) and prediction tasks.
• Present an encoder–decoder framework to categorize dynamic-graph models.
• Survey temporal encoding/decoding techniques including sequence models, attention, and temporal point processes.
• Discuss expressivity notions for dynamic-graph models and KG/ HIN extensions.
• Review datasets and applications, and outline open problems and future directions.

Experimental results

Research questions

• RQ1 What are the main encoder and decoder techniques used for dynamic graphs and how are they categorized?
• RQ2 How do dynamic graphs handle interpolation and extrapolation tasks, including streaming scenarios?
• RQ3 What are the key datasets, applications, and open challenges in dynamic graph representation learning?
• RQ4 How is expressivity defined and analyzed for dynamic-graph models, including knowledge graphs and heterogeneous information networks?

Key findings

• The survey formalizes dynamic graphs via continuous-time and discrete-time definitions and clarifies prediction tasks like node/edge/graph classification.
• An encoder–decoder perspective is used to systematize dynamic-graph models and their embeddings across time.
• It covers temporal encoding methods, sequence models, attention mechanisms, and temporal point processes as core techniques.
• The work discusses model expressivity, including full expressivity notions for node, edge, and graph prediction tasks.
• Applications, datasets, and open research directions are summarized to guide future work in dynamic graph representation learning.