[Paper Review] Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking

Aleksandar Bojchevski, Stephan Günnemann|arXiv (Cornell University)|Jul 12, 2017

Advanced Graph Neural Networks32 references188 citations

TL;DR

Graph2Gauss learns node embeddings as Gaussian distributions in an unsupervised, inductive framework, using a personalized ranking objective over multi-hop neighborhoods to capture uncertainty and network structure.

ABSTRACT

Methods that learn representations of nodes in a graph play a critical role in network analysis since they enable many downstream learning tasks. We propose Graph2Gauss - an approach that can efficiently learn versatile node embeddings on large scale (attributed) graphs that show strong performance on tasks such as link prediction and node classification. Unlike most approaches that represent nodes as point vectors in a low-dimensional continuous space, we embed each node as a Gaussian distribution, allowing us to capture uncertainty about the representation. Furthermore, we propose an unsupervised method that handles inductive learning scenarios and is applicable to different types of graphs: plain/attributed, directed/undirected. By leveraging both the network structure and the associated node attributes, we are able to generalize to unseen nodes without additional training. To learn the embeddings we adopt a personalized ranking formulation w.r.t. the node distances that exploits the natural ordering of the nodes imposed by the network structure. Experiments on real world networks demonstrate the high performance of our approach, outperforming state-of-the-art network embedding methods on several different tasks. Additionally, we demonstrate the benefits of modeling uncertainty - by analyzing it we can estimate neighborhood diversity and detect the intrinsic latent dimensionality of a graph.

Motivation & Objective

• Motivate node representation learning for graphs with uncertainty and inductive generalization.
• Propose a Gaussian embedding for nodes to capture uncertainty.
• Develop an unsupervised personalized ranking objective over multi-hop neighborhoods.
• Enable inductive learning by mapping node attributes to embeddings via a deep encoder.

Proposed method

• Represent each node i as a Gaussian N(mu_i, Sigma_i) in a low-dimensional space.
• Use a deep encoder to map node attributes x_i to mu_i and Sigma_i (diagonal).
• Define an asymmetric KL divergence delta(hi,hj)=DKL(Nj||Ni) as a dissimilarity between embeddings.
• Impose a personalized ranking: nodes at 1-hop should be closer than 2-hop, etc., up to K hops.
• Optimize a square-exponential loss that compares energies E_ijk=DKL(N_jk||N_i) and E_ijl=DKL(N_jl||N_i) over valid triplets.
• Adopt node-anchored stochastic sampling to ensure unbiased gradient estimates and scalable training.
• Leverage attribute information to enable inductive generalization to unseen nodes.
• Support plain graphs by using one-hot encodings when attributes are absent.

Experimental results

Research questions

• RQ1 Can nodes be effectively embedded as distributions to capture uncertainty in attributed graphs?
• RQ2 Does a personalized ranking over multi-hop neighborhoods improve unsupervised graph embeddings?
• RQ3 Can the encoder generalize to unseen nodes using only node attributes (inductive learning)?
• RQ4 How does the proposed method compare to state-of-the-art unsupervised graph embeddings on link prediction and node classification?
• RQ5 What insights does embedding uncertainty provide about neighborhood diversity and intrinsic dimensionality?

Key findings

• Graph2Gauss achieves state-of-the-art or competitive link prediction performance across multiple real-world datasets (Cora-ML, Cora, Citeseer, DBLP, PubMed).
• The method outperforms competitors on average AUC and AP metrics for link prediction with embedding size L=128.
• Even the attribute-less variant G2Goh can outperform several baselines on some datasets.
• Graph2Gauss demonstrates strong node classification performance on Cora-ML, Citeseer, and DBLP with unsupervised pretraining.
• The model provides meaningful uncertainty measures that correlate with neighborhood diversity and helps reveal intrinsic latent dimensionality.