[Paper Review] GraphGAN: Graph Representation Learning with Generative Adversarial Nets

Hongwei Wang, Jia Wang+6 more|arXiv (Cornell University)|Nov 22, 2017

Advanced Graph Neural Networks34 references178 citations

TL;DR

GraphGAN unifies generative and discriminative graph representation learning via a GAN framework, introducing graph softmax and a BFS-based online generator to improve link prediction, node classification, and recommendation.

ABSTRACT

The goal of graph representation learning is to embed each vertex in a graph into a low-dimensional vector space. Existing graph representation learning methods can be classified into two categories: generative models that learn the underlying connectivity distribution in the graph, and discriminative models that predict the probability of edge existence between a pair of vertices. In this paper, we propose GraphGAN, an innovative graph representation learning framework unifying above two classes of methods, in which the generative model and discriminative model play a game-theoretical minimax game. Specifically, for a given vertex, the generative model tries to fit its underlying true connectivity distribution over all other vertices and produces "fake" samples to fool the discriminative model, while the discriminative model tries to detect whether the sampled vertex is from ground truth or generated by the generative model. With the competition between these two models, both of them can alternately and iteratively boost their performance. Moreover, when considering the implementation of generative model, we propose a novel graph softmax to overcome the limitations of traditional softmax function, which can be proven satisfying desirable properties of normalization, graph structure awareness, and computational efficiency. Through extensive experiments on real-world datasets, we demonstrate that GraphGAN achieves substantial gains in a variety of applications, including link prediction, node classification, and recommendation, over state-of-the-art baselines.

Motivation & Objective

• Motivate unifying generative and discriminative graph representation learning within a single adversarial framework.
• Propose GraphGAN with a generator aiming to fit the true connectivity distribution and a discriminator distinguishing true vs generated edges.
• Introduce graph softmax to overcome inefficiencies of traditional softmax and capture graph structure.
• Develop an online sampling strategy for efficient generation that respects graph proximity.
• Evaluate GraphGAN on multiple real-world graph datasets across three tasks to demonstrate performance gains.

Proposed method

• Formulate GraphGAN as a two-player minimax game between a generator G(v|v_c) and a discriminator D(v,v_c).
• Discriminator D uses a sigmoid of vertex embeddings inner product: D(v,v_c)=sigmoid(d_v^T d_{v_c}).
• Generator G aims to approximate the true connectivity distribution p_true(v|v_c) and generate likely neighbors.
• Introduce graph softmax G(v|v_c) with a BFS-tree T_c and path-based product of local softmaxes to ensure normalization, structure awareness, and efficiency.
• Provide an online generating strategy that samples via a random walk on T_c with transition probabilities p_c(v_i|v) defined by local softmax over neighbors.
• Derive gradients for G with policy gradient due to discrete sampling, enabling updating θ_G via expected log-probabilities weighted by discriminator feedback.

Experimental results

Research questions

• RQ1 Can generative and discriminative graph representation learning be effectively unified under a minimax adversarial framework?
• RQ2 Does graph softmax provide normalization, graph-structure awareness, and computational efficiency over standard softmax, hierarchical softmax, or negative sampling in graphs?
• RQ3 Can an online, BFS-based generator sampling strategy achieve scalable training for large graphs without sacrificing accuracy?
• RQ4 Do GraphGAN embeddings improve performance on link prediction, node classification, and recommendation compared with strong baselines?

Key findings

• GraphGAN outperforms strong baselines in link prediction on arXiv-AstroPh and arXiv-GrQc, with accuracy gains ranging from 0.59% to 11.13% and Macro-F1 gains from 0.59% to 11.13% (depending on dataset).
• GraphGAN improves node classification on BlogCatalog and Wikipedia, with accuracy gains from 0.95% to 21.71% and Macro-F1 gains up to 21.71%.
• In recommendation on MovieLens-1M, GraphGAN yields Precision@20 and Recall@20 improvements, e.g., Precision@20 surpassing baselines by 38.56% and Recall@20 by 52.33%.
• The learning dynamics show the generator reaching strong performance while the discriminator remains informative but not at random chance, indicating a stable equilibrium in the minimax game.
• Graph softmax empirically captures the proximity-aware connectivity pattern of graphs and reduces computation from full softmax to O(d log V) per sample.