[Paper Review] Adversarially Regularized Graph Autoencoder for Graph Embedding

Shirui Pan, Ruiqi Hu+4 more|arXiv (Cornell University)|Feb 13, 2018

Advanced Graph Neural Networks24 references70 citations

TL;DR

ARGA and ARVGA jointly learn graph embeddings by combining a graph convolutional autoencoder with adversarial regularization to match a prior distribution, improving link prediction, clustering, and visualization over baselines.

ABSTRACT

Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics. Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data, but they have mostly ignored the data distribution of the latent codes from the graphs, which often results in inferior embedding in real-world graph data. In this paper, we propose a novel adversarial graph embedding framework for graph data. The framework encodes the topological structure and node content in a graph to a compact representation, on which a decoder is trained to reconstruct the graph structure. Furthermore, the latent representation is enforced to match a prior distribution via an adversarial training scheme. To learn a robust embedding, two variants of adversarial approaches, adversarially regularized graph autoencoder (ARGA) and adversarially regularized variational graph autoencoder (ARVGA), are developed. Experimental studies on real-world graphs validate our design and demonstrate that our algorithms outperform baselines by a wide margin in link prediction, graph clustering, and graph visualization tasks.

Motivation & Objective

• Motivate the need for robust graph embeddings that respect latent data distributions, not just structure reconstruction.
• Propose two adversarially regularized graph embedding models (ARGA and ARVGA) that integrate structure, content, and prior-latent distributions.
• Demonstrate superior performance on unsupervised tasks: link prediction, node clustering, and graph visualization.

Proposed method

• Encode graph structure and node content with a graph convolutional encoder to obtain latent Z.
• Decode to reconstruct the graph adjacency A via a link-prediction layer from Z.
• Regularize latent codes by adversarially forcing Z to match a prior distribution (Gaussian) using a discriminator.
• Train ARGA (graph autoencoder) and ARVGA (variational graph autoencoder) in a joint optimization with the adversarial module.
• Optimization involves L0 (reconstruction), L1 (variational lower bound), and adversarial loss (Eq. 12 and 13).
• Encoder variants include a two-layer GCN, with Z = q(Z|X,A) and Gaussian prior p(Z).

Experimental results

Research questions

• RQ1 Can adversarial regularization improve the quality of graph embeddings beyond reconstruction-based objectives?
• RQ2 Do ARGA and ARVGA produce embeddings that enhance link prediction, clustering, and visualization compared to existing graph embedding methods?
• RQ3 How does incorporating both graph structure and node content via GCN affect downstream unsupervised tasks?
• RQ4 What is the impact of using a variational encoder (ARVGA) versus a standard graph autoencoder (ARGA)?

Key findings

• ARGA and ARVGA achieve high link-prediction performance on benchmark graphs, with AP and AUC scores around 92% across Cora, Citeseer, and PubMed datasets.
• On clustering, ARGA/ARVGA substantially improve Acc, NMI, F1, Precision, and ARI over baselines on Cora and Citeseer.
• Visualizations (t-SNE) of learned embeddings show more meaningful layouts for ARGA/ARVGA compared with baselines.
• Compared to VGAE and GAE variants, ARGA/ARVGA significantly outperform in both link prediction and clustering tasks (as reported in Table 2 and Tables 3-4).
• Table 2 reports dataset-wide AUC and AP improvements for ARGA/ARVGA over baselines (e.g., on PubMed: ARGE AUC 96.8, ARGE AP 97.1; ARVGE AUC 96.5, ARVGE AP 96.8).