[Paper Review] A Comprehensive Survey on Deep Graph Representation Learning

Wei Ju, Zheng Fang+14 more|arXiv (Cornell University)|Apr 11, 2023

Advanced Graph Neural Networks563 references14 citations

TL;DR

A systematic taxonomy and comprehensive review of deep graph representation learning, covering GNN architectures, learning paradigms, and applications.

ABSTRACT

Graph representation learning aims to effectively encode high-dimensional sparse graph-structured data into low-dimensional dense vectors, which is a fundamental task that has been widely studied in a range of fields, including machine learning and data mining. Classic graph embedding methods follow the basic idea that the embedding vectors of interconnected nodes in the graph can still maintain a relatively close distance, thereby preserving the structural information between the nodes in the graph. However, this is sub-optimal due to: (i) traditional methods have limited model capacity which limits the learning performance; (ii) existing techniques typically rely on unsupervised learning strategies and fail to couple with the latest learning paradigms; (iii) representation learning and downstream tasks are dependent on each other which should be jointly enhanced. With the remarkable success of deep learning, deep graph representation learning has shown great potential and advantages over shallow (traditional) methods, there exist a large number of deep graph representation learning techniques have been proposed in the past decade, especially graph neural networks. In this survey, we conduct a comprehensive survey on current deep graph representation learning algorithms by proposing a new taxonomy of existing state-of-the-art literature. Specifically, we systematically summarize the essential components of graph representation learning and categorize existing approaches by the ways of graph neural network architectures and the most recent advanced learning paradigms. Moreover, this survey also provides the practical and promising applications of deep graph representation learning. Last but not least, we state new perspectives and suggest challenging directions which deserve further investigations in the future.

Motivation & Objective

• Provide a taxonomy of deep graph representation learning methods organized by GNN architectures and learning paradigms.
• Summarize essential components and representative algorithms within each category.
• Highlight practical applications and discuss challenges, limitations, and future directions.

Proposed method

• Classify graph representation learning approaches by graph neural network architectures (graph convolutions, graph kernel neural networks, graph pooling, graph transformers).
• Examine learning paradigms including supervised/semi-supervised, self-supervised, and graph structure learning.
• Discuss representative algorithms and summarize their characteristics in each category.
• Present applications in social analysis, molecular property prediction and generation, recommender systems, and traffic analysis.
• Offer insights into future directions and open challenges.

Experimental results

Research questions

• RQ1 What are the main GNN architectures used for deep graph representation learning and their key operations?
• RQ2 How do different learning paradigms (supervised/semi-supervised, self-supervised, structure learning) influence graph representations?
• RQ3 What are the current applications that benefit most from deep graph representations and what challenges remain?
• RQ4 What future directions and open problems are most promising for advancing deep graph representation learning?

Key findings

• Provides a new taxonomy of deep graph representation learning methods based on GNN architectures and learning paradigms.
• Offers a comprehensive review of essential components and representative algorithms in each branch.
• Discusses promising applications such as social analysis, molecular property prediction and generation, recommender systems, and traffic analysis.
• Identifies limitations and challenging directions for future research.