[Paper Review] Link Prediction Based on Graph Neural Networks

Muhan Zhang|arXiv (Cornell University)|Feb 27, 2018

Complex Network Analysis Techniques46 references282 citations

TL;DR

The paper introduces SEAL, a framework that learns link prediction heuristics from local enclosing subgraphs using GNNs, backed by a gamma-decaying theory that unifies high-order heuristics.

ABSTRACT

Link prediction is a key problem for network-structured data. Link prediction heuristics use some score functions, such as common neighbors and Katz index, to measure the likelihood of links. They have obtained wide practical uses due to their simplicity, interpretability, and for some of them, scalability. However, every heuristic has a strong assumption on when two nodes are likely to link, which limits their effectiveness on networks where these assumptions fail. In this regard, a more reasonable way should be learning a suitable heuristic from a given network instead of using predefined ones. By extracting a local subgraph around each target link, we aim to learn a function mapping the subgraph patterns to link existence, thus automatically learning a `heuristic' that suits the current network. In this paper, we study this heuristic learning paradigm for link prediction. First, we develop a novel $γ$-decaying heuristic theory. The theory unifies a wide range of heuristics in a single framework, and proves that all these heuristics can be well approximated from local subgraphs. Our results show that local subgraphs reserve rich information related to link existence. Second, based on the $γ$-decaying theory, we propose a new algorithm to learn heuristics from local subgraphs using a graph neural network (GNN). Its experimental results show unprecedented performance, working consistently well on a wide range of problems.

Motivation & Objective

• Motivate learning general graph-structure based heuristics for link prediction beyond predefined measures.
• Show that local enclosing subgraphs contain sufficient information to approximate high-order heuristics.
• Propose SEAL, a GNN-based framework that integrates subgraphs, node embeddings, and attributes for improved prediction.
• Provide theoretical justification via gamma-decaying heuristics and demonstrate empirical superiority over baselines.

Proposed method

• Define enclosing subgraphs around target links to capture local structure.
• Prove gamma-decaying heuristic theory showing approximation from h-hop subgraphs with exponentially decreasing error.
• Replace WLNM’s fully-connected network with a graph neural network (GNN) to learn from subgraphs.
• Incorporate three node feature components: structural labels, node embeddings, and explicit attributes (via DRNL labeling and embedding tricks).
• Use negative injection to prevent leakage from training links when generating node embeddings.
• Train SEAL on positive and negative examples to predict link existence using a graph-level representation of subgraphs.

Experimental results

Research questions

• RQ1 Can local h-hop enclosing subgraphs sufficiently capture information needed to predict link existence, including high-order effects?
• RQ2 Do gamma-decaying heuristics provide a unified, provable basis for approximating common high-order link predictors from local subgraphs?
• RQ3 Does SEAL outperform traditional heuristics, latent-feature methods, and prior supervised subgraph approaches for link prediction across diverse networks?
• RQ4 What is the impact of incorporating latent/explicit features and node labeling on SEAL’s performance?

Key findings

• SEAL generally outperforms predefined heuristics, WLK, and WLNM across multiple datasets.
• SEAL with GNNs achieves state-of-the-art results among learning-based methods, surpassing latent feature approaches like MF, SBM, N2V, LINE, SPC, and VGAE.
• Incorporating node embeddings along with structural features yields significant gains over structure-only baselines.
• The gamma-decaying theory demonstrates that many high-order heuristics can be approximated from small enclosing subgraphs with exponentially decreasing error.
• Using DRNL labeling and negative injection improves SEAL’s learning and generalization.
• SEAL’s performance remains strong when using different GNN architectures and embeddings, highlighting robustness.