[Paper Review] Representation Learning on Graphs: Methods and Applications

William L. Hamilton, Rex Ying+1 more|arXiv (Cornell University)|Sep 17, 2017

Advanced Graph Neural NetworksComputer Science298 citations

TL;DR

A comprehensive survey of graph representation learning, unifying node and subgraph embeddings under an encoder–decoder framework and detailing shallow, deep, and neighborhood-aggregation methods.

ABSTRACT

Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph (e.g., degree statistics or kernel functions). However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. Here we provide a conceptual review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph neural networks. We review methods to embed individual nodes as well as approaches to embed entire (sub)graphs. In doing so, we develop a unified framework to describe these recent approaches, and we highlight a number of important applications and directions for future work.

Motivation & Objective

• Explain the core problem of encoding graph structure into low-dimensional embeddings for downstream tasks.
• Unify diverse graph embedding approaches under a common encoder–decoder framework.
• Review scalable methods for large graphs and discuss node and subgraph representations and their applications.
• Highlight limitations, applications, and future directions in graph representation learning.

Proposed method

• Introduce a unified encoder–decoder framework for node embeddings.
• Define a generic pairwise decoder and graph-based similarity objective to reconstruct graph structure from embeddings.
• categorize methods into shallow embeddings (matrix factorization and random walks) and generalized encoder architectures.
• Discuss neighborhood aggregation/convolutional encoders that use node attributes and local neighborhoods.
• Address optimization strategies and practical considerations for large graphs.

Experimental results

Research questions

• RQ1 How can graph structure be encoded into low-dimensional embeddings that preserve graph topology and node roles?
• RQ2 What are the main families of node-embedding methods and how do they differ in their encoder/decoder choices and loss functions?
• RQ3 How can we move beyond shallow embeddings to handle dynamic, large-scale, or attributed graphs?
• RQ4 What are the trade-offs between first-order and higher-order neighborhood representations in practice?
• RQ5 What are the key directions and limitations in graph representation learning for future work?

Key findings

• There is a unifying encoder–decoder perspective that groups node embedding methods by their encoder, decoder, similarity measure, and loss.
• Shallow embeddings rely on matrix factorization or random walks and can be scaled to large graphs, but have transductive limitations.
• Random-walk based methods (DeepWalk, node2vec) leverage biased walks to capture community structure or structural roles.
• Generalized encoder architectures (neighborhood aggregation/convolutional encoders) and deep models address limitations of shallow methods by incorporating node attributes and graph locality.
• Neighborhood aggregation methods enable inductive representation learning, enabling embeddings for unseen nodes and evolving graphs.
• A common framework helps compare seemingly disparate methods and highlights practical considerations for scalability and applicability.