[Paper Review] Variational Recurrent Neural Networks for Graph Classification

Edouard Pineau, Nathan de Lara|arXiv (Cornell University)|Feb 7, 2019

Advanced Graph Neural NetworksComputer Science24 references3 citations

TL;DR

This paper proposes a variational recurrent graph classifier (VRGC) that sequentially processes graphs using BFS-based node ordering and gated recurrent units (GRUs) to learn invariant representations, with a novel variational auto-regressive block for structural regularization. The model achieves state-of-the-art performance on four molecular graph datasets, particularly excelling on NCI1 and Proteins Full.

ABSTRACT

We address the problem of graph classification based only on structural information. Inspired by natural language processing techniques (NLP), our model sequentially embeds information to estimate class membership probabilities. Besides, we experiment with NLP-like variational regularization techniques, making the model predict the next node in the sequence as it reads it. We experimentally show that our model achieves state-of-the-art classification results on several standard molecular datasets. Finally, we perform a qualitative analysis and give some insights on whether the node prediction helps the model better classify graphs.

Motivation & Objective

• To address the challenge of graph classification with variable-sized, ordering-dependent inputs using sequence modeling techniques from NLP.
• To improve generalization and structural representation learning in graph neural networks without relying on node or edge attributes.
• To investigate whether variational auto-regressive regularization enhances classification performance by encouraging the model to predict node connectivity during training.
• To achieve node-indexing invariance through data augmentation via random BFS roots during training and testing.
• To evaluate the contribution of the variational auto-regression block to the final classification accuracy on standard molecular benchmarks.

Proposed method

• The model applies a random BFS traversal to reorder nodes in an undirected graph, transforming the adjacency matrix into a sequence of d-dimensional node feature vectors.
• Each node's adjacency vector is truncated to the first d preceding nodes in the BFS order and zero-padded to maintain fixed dimensionality.
• A GRU processes the sequence of node embeddings, learning a latent representation that captures structural dependencies across the graph.
• A separate GRU and softmax classifier head use the final hidden state to predict graph class probabilities via cross-entropy loss.
• A variational auto-regressive (VAR) block is introduced to predict the next node’s adjacency vector at each step, providing auxiliary training signal and regularization.
• During inference, the model aggregates predictions over N random BFS roots and applies a hard vote over K runs to improve robustness and confidence in predictions.

Experimental results

Research questions

• RQ1 Does incorporating variational auto-regressive regularization improve graph classification performance compared to standard recurrent graph classifiers?
• RQ2 To what extent does the use of random BFS-based node ordering contribute to learning node-indexing invariant representations?
• RQ3 How does the VAR block’s auxiliary prediction task influence the quality of the learned latent representation for classification?
• RQ4 Can the proposed method achieve state-of-the-art results on standard molecular graph benchmarks without using node or edge attributes?
• RQ5 Does the combination of data augmentation via BFS root sampling and ensemble averaging improve generalization and prediction confidence?

Key findings

• The VRGC model achieves state-of-the-art accuracy on three out of four standard molecular graph datasets: 86.3% on Mutag, 48.4% on Enzymes, 74.8% on Proteins Full, and 80.7% on NCI1.
• On the NCI1 dataset, VRGC outperforms all prior methods, including the previous SOTA (FGSD at 79.8%), achieving 80.7% accuracy.
• The variational auto-regressive block contributes to improved performance, particularly on larger datasets like NCI1 and Proteins Full, suggesting it enhances structural representation learning.
• The model demonstrates robustness to node indexing through data augmentation: multiple BFS traversals per graph during inference lead to stable and confident predictions.
• Qualitative analysis shows that the latent space clusters corresponding to the same graph (under different node orderings) form tight, coherent groups, indicating effective invariance learning.
• The additional training cost of the VAR block is marginal, making it an efficient regularization technique for graph classification.