[Paper Review] Inductive Representation Learning on Large Graphs

William L. Hamilton, Rex Ying+1 more|arXiv (Cornell University)|Jun 7, 2017

Bioinformatics and Genomic Networks34 references4,534 citations

TL;DR

GraphSAGE presents an inductive framework to generate node embeddings for unseen nodes by learning neighborhood aggregation functions over node features, outperforming baselines on three inductive node-classification tasks.

ABSTRACT

Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.

Motivation & Objective

• Motivate the need for inductive node embeddings that generalize to unseen nodes in evolving graphs.
• Propose a general GraphSAGE framework that learns to aggregate neighborhood features to generate node embeddings.
• Evaluate multiple aggregator architectures and demonstrate improved predictive performance over baselines on diverse datasets.
• Show that the method can generalize across graphs and provide theoretical insight into learning local graph structure.

Proposed method

• Propose GraphSAGE, which learns trainable aggregator functions to combine features from a node’s local neighborhood and generate embeddings via multiple hops (K).
• Use a forward propagation procedure (Algorithm 1) where each layer aggregates neighbor representations, concatenates with the node’s own representation, and applies a nonlinear transformation with learned weights W^k.
• Adopt unsupervised loss (Equation 1) encouraging nearby nodes to have similar representations and distant nodes to be distinct, with optional supervised variants for task-specific objectives.
• Explore different aggregator architectures (mean, LSTM, pooling) to capture neighborhood information while ensuring symmetry with respect to neighbor order.

Experimental results

Research questions

• RQ1 Can GraphSAGE generate meaningful embeddings for nodes not seen during training (inductive setting)?
• RQ2 How do different neighborhood aggregators affect inductive embedding quality and scalability?
• RQ3 To what extent can the learned aggregators capture local graph structure and enable cross-graph generalization?
• RQ4 How does GraphSAGE compare to transductive baselines and other embedding methods on real-world inductive tasks?

Key findings

• GraphSAGE outperforms baselines (random, raw features, DeepWalk, and DeepWalk+features) across citation, Reddit, and PPI datasets.
• Among aggregators, LSTM and pooling variants deliver the strongest performance overall, with mean offering competitive results and GCN-based aggregation showing weaker performance in some tasks.
• Unsupervised GraphSAGE can achieve strong performance close to fully supervised variants, indicating good utility without task-specific labels.
• GraphSAGE variants with K=2 and moderate neighborhood sampling provide significant accuracy gains (roughly 10-15% average over K=1) while offering favorable runtimes.
• The approach demonstrates cross-graph generalization in the PPI setting, where training on multiple graphs improves performance on unseen graphs.