[Paper Review] Deep Learning on Graphs: A Survey

Ziwei Zhang, Peng Cui+1 more|arXiv (Cornell University)|Dec 11, 2018

Advanced Graph Neural Networks170 references168 citations

TL;DR

A comprehensive survey categorizing deep learning methods on graphs into five architectures, analyzing their differences, and outlining applications and future directions.

ABSTRACT

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Motivation & Objective

• Motivate the study of deep learning on graphs and address the challenges posed by graph data (irregular structures, heterogeneity, scale, and interdisciplinary knowledge).
• Provide a systematic review of existing graph-based deep learning methods organized by model architecture and training strategy.
• Clarify relationships and compositions among Graph RNNs, GCNs, GAEs, Graph RL, and graph adversarial methods.
• Discuss applications, libraries, and potential future research directions in graph-based deep learning.

Proposed method

• Divide existing methods into five categories based on architecture and training strategy: Graph RNNs, Graph Convolutional Networks, Graph Autoencoders, Graph Reinforcement Learning, and Graph Adversarial Methods.
• Review developments in each category in a roughly historical order to highlight progression and connections.
• Analyze differences and combinations across methods to reveal underlying unifying principles.
• Provide an overview of applications and discuss future research directions and potential libraries.
• Include supplementary materials such as an appendix with code repositories and time complexity analyses.

Experimental results

Research questions

• RQ1 What are the main deep learning architectures used on graphs and how do they differ conceptually and computationally?
• RQ2 How can different graph-based models be related or composed to solve broader graph tasks?
• RQ3 What are the primary challenges in applying deep learning to graph data and how have methods addressed them?
• RQ4 What are the common applications and future directions for graph-based deep learning?

Key findings

• The survey identifies five core categories of graph-based deep learning methods: Graph RNNs, GCNs, GAEs, Graph RL, and graph adversarial methods.
• It analyzes the evolution and connections among these methods, highlighting how many GNNs relate to recurrent frameworks and how spectral and spatial approaches converge.
• It discusses scalability challenges and architectural solutions to handle large-scale graphs and heterogeneous graph types.
• It notes the interdisciplinary nature of graph data and how domain knowledge informs model design and constraints.
• It provides an overview of applications and points to open libraries and future research directions.