[Paper Review] Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Ines Chami, Sami Abu-El-Haija+3 more|arXiv (Cornell University)|May 7, 2020

Advanced Graph Neural NetworksComputer Science150 references99 citations

TL;DR

This survey unifies network embedding, graph regularization, and graph neural networks under the GraphEDM framework and provides a comprehensive taxonomy of Graph Representation Learning methods, plus an open-source library."

ABSTRACT

There has been a surge of recent interest in learning representations for graph-structured data. Graph representation learning methods have generally fallen into three main categories, based on the availability of labeled data. The first, network embedding (such as shallow graph embedding or graph auto-encoders), focuses on learning unsupervised representations of relational structure. The second, graph regularized neural networks, leverages graphs to augment neural network losses with a regularization objective for semi-supervised learning. The third, graph neural networks, aims to learn differentiable functions over discrete topologies with arbitrary structure. However, despite the popularity of these areas there has been surprisingly little work on unifying the three paradigms. Here, we aim to bridge the gap between graph neural networks, network embedding and graph regularization models. We propose a comprehensive taxonomy of representation learning methods for graph-structured data, aiming to unify several disparate bodies of work. Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which generalizes popular algorithms for semi-supervised learning on graphs (e.g. GraphSage, Graph Convolutional Networks, Graph Attention Networks), and unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc) into a single consistent approach. To illustrate the generality of this approach, we fit over thirty existing methods into this framework. We believe that this unifying view both provides a solid foundation for understanding the intuition behind these methods, and enables future research in the area.

Motivation & Objective

• Propose a unified framework (GraphEDM) to describe both supervised and unsupervised graph representation learning methods.
• Provide a comprehensive taxonomy that subsumes dozens of existing GRL approaches (shallow embeddings, auto-encoders, graph regularization, and GNNs).
• Introduce a Graph Convolution Framework (GCF) to analyze convolution-based GNNs across spectral and spatial paradigms.
• Position the work within the broader context of Graph Neural Networks and non-Euclidean learning, outlining advantages and limitations.
• Release an open-source GRL library to enable replication and application to node classification and link prediction tasks.

Proposed method

• Introduce GraphEDM as an encoder-decoder model that maps graph structure and optional node features to node embeddings, a reconstructed similarity matrix, and optional label predictions.
• Define a loss for GraphEDM as a weighted sum of supervised loss, graph-regularization loss, and parameter regularization (L2), enabling both supervised and unsupervised training.
• Enumerate a taxonomy of objective functions, including supervised losses, graph-regularization losses, and regularization penalties, with a total loss L = alpha*L_SUP + beta*L_G,REG + gamma*L_REG.
• Categorize encoders into four groups: shallow embeddings, graph regularization, graph auto-encoding, and neighborhood aggregation methods (graph convolutional approaches).
• Describe how GraphEDM can realize existing methods (e.g., GraphSAGE, GCN, GAT, DeepWalk, node2vec) by specific encoder/decoder choices.
• Provide a high-level architecture diagram and discuss transductive vs inductive settings, feature usage, and positional vs structural embeddings.

Experimental results

Research questions

• RQ1 How can network embedding, graph regularization, and graph neural networks be unified into a single descriptive framework?
• RQ2 What is a comprehensive taxonomy that subsumes a broad set of GRL methods and clarifies their similarities and differences?
• RQ3 How can GraphSAGE, GCN, GAT, DeepWalk, and node2vec be represented within a common GraphEDM/Graph Convolution Framework?
• RQ4 What insights and limitations arise when describing supervised vs unsupervised, transductive vs inductive, and positional vs structural embeddings under one framework?
• RQ5 Can an open-source library support broad GRL methods and common applications like node classification and link prediction?

Key findings

• The GraphEDM framework generalizes popular semi-supervised and unsupervised GRL methods within a single encoder-decoder perspective.
• The authors categorize over thirty existing GRL methods using a unified taxonomy, clarifying their relationships and differences.
• The Graph Convolution Framework (GCF) is introduced to analyze convolution-based GNNs spanning spectral to spatial and attention-based methods.
• The work demonstrates how various methods (shallow embeddings, auto-encoders, graph regularization, and GNNs) fit into GraphEDM, enabling principled comparisons.
• An open-source GRL library is released to support state-of-the-art methods and common graph applications such as node classification and link prediction.