[Paper Review] Learning Graph Representations with Embedding Propagation

Alberto García-Durán, Mathias Niepert|arXiv (Cornell University)|Oct 9, 2017

Advanced Graph Neural NetworksComputer Science33 references75 citations

TL;DR

The paper introduces Embedding Propagation (Ep), an unsupervised framework that learns graph embeddings by passing label-based messages between neighboring nodes and reconstructing label embeddings to update node representations. Ep-B, a simple Ep instance, achieves competitive or superior performance on multiple graph benchmarks with fewer parameters and hyperparameters.

ABSTRACT

We propose Embedding Propagation (EP), an unsupervised learning framework for graph-structured data. EP learns vector representations of graphs by passing two types of messages between neighboring nodes. Forward messages consist of label representations such as representations of words and other attributes associated with the nodes. Backward messages consist of gradients that result from aggregating the label representations and applying a reconstruction loss. Node representations are finally computed from the representation of their labels. With significantly fewer parameters and hyperparameters an instance of EP is competitive with and often outperforms state of the art unsupervised and semi-supervised learning methods on a range of benchmark data sets.

Motivation & Objective

• Motivate unsupervised learning for graph-structured data across domains like social networks and bioinformatics.
• Propose Embedding Propagation (Ep) to learn node representations by exchanging label embeddings between neighbors.
• Enable reconstruction-based learning that updates label embeddings through gradient signals from reconstruction losses.
• Provide a concrete Ep-B instance with simple averaging aggregators and margin-based ranking loss to demonstrate effectiveness.

Proposed method

• Embed each type of node label with a differentiable function f_i to obtain label embeddings.
• Compute vertex embeddings by aggregating label embeddings via functions g_i and reconstruct them via ~g_i from neighbor label embeddings.
• Minimize a reconstruction loss across all label types to align current and reconstructed label embeddings.
• Use a margin-based ranking loss to encourage closer reconstruction of a node's labels to its own embeddings than to others' embeddings.
• Concatenate per-type label embeddings to form the node representation that feeds downstream tasks.

Experimental results

Research questions

• RQ1 Can unsupervised embedding propagation learn competitive graph representations without supervised labels?
• RQ2 How effectively can multi-type node labels (e.g., words, IDs, attributes) be integrated and propagated across a graph?
• RQ3 Does Ep-B with simple averaging and a margin loss scale to real-world graphs and support inductive learning?

Key findings

• Ep-B achieves best or near-best micro and macro F1 scores on BlogCatalog, POS, and PPI in both transductive and inductive settings.
• Ep-B consistently outperforms baselines like DeepWalk, Node2vec, LINE, and wvRN on several datasets, with fewer parameters and hyperparameters to tune.
• Ep-B remains effective when edges are directed or when graphs have multiple label types and attribute information.
• Embedding visualization shows Ep-B effectively combines multiple label types to form separable clusters.
• Ep representations support inductive generalization by reconstructing label embeddings for unseen nodes without retraining.