[Paper Review] A Flexible Generative Framework for Graph-based Semi-supervised Learning

Jiaqi Ma, Weijing Tang+2 more|arXiv (Cornell University)|May 26, 2019

Advanced Graph Neural Networks24 citations

TL;DR

This paper proposes a flexible generative framework for graph-based semi-supervised learning that models the joint distribution of node features, labels, and graph structure using probabilistic graphical models. By leveraging scalable variational inference and random graph model insights, the method outperforms state-of-the-art models on benchmark datasets, demonstrating improved label prediction accuracy through better integration of feature, label, and structural dependencies.

ABSTRACT

We consider a family of problems that are concerned about making predictions for the majority of unlabeled, graph-structured data samples based on a small proportion of labeled samples. Relational information among the data samples, often encoded in the graph/network structure, is shown to be helpful for these semi-supervised learning tasks. However, conventional graph-based regularization methods and recent graph neural networks do not fully leverage the interrelations between the features, the graph, and the labels. In this work, we propose a flexible generative framework for graph-based semi-supervised learning, which approaches the joint distribution of the node features, labels, and the graph structure. Borrowing insights from random graph models in network science literature, this joint distribution can be instantiated using various distribution families. For the inference of missing labels, we exploit recent advances of scalable variational inference techniques to approximate the Bayesian posterior. We conduct thorough experiments on benchmark datasets for graph-based semi-supervised learning. Results show that the proposed methods outperform state-of-the-art models under most settings.

Motivation & Objective

• To address the challenge of predicting labels for a large number of unlabeled nodes in graph-structured data using only a small set of labeled examples.
• To overcome the limitations of conventional graph regularization and recent GNNs in fully exploiting interrelations among features, labels, and graph structure.
• To develop a unified probabilistic framework that models the joint distribution of features, labels, and graph topology in a flexible and scalable manner.
• To enable effective inference of missing labels through scalable variational inference techniques grounded in Bayesian principles.
• To demonstrate the superiority of the proposed framework over existing state-of-the-art models across diverse graph-based semi-supervised learning benchmarks.

Proposed method

• The framework models the joint distribution of node features, labels, and graph structure using a probabilistic graphical model inspired by random graph models from network science.
• It allows instantiation with various distribution families, enabling flexibility in modeling different types of relational and feature dependencies in the graph.
• Scalable variational inference is employed to approximate the intractable Bayesian posterior over labels given observed features and graph structure.
• The method jointly optimizes over the parameters of the generative model and the variational approximation, enabling end-to-end learning.
• The framework supports both transductive and inductive learning settings by modeling the full joint distribution over nodes and their attributes.
• It leverages the relational structure of the graph not only as a regularizer but as a core component of the generative process for labels and features.

Experimental results

Research questions

• RQ1 Can a unified generative model that jointly captures features, labels, and graph structure outperform discriminative GNNs and conventional graph regularization?
• RQ2 How effectively can scalable variational inference be applied to approximate the posterior over labels in a complex, high-dimensional graph-structured joint distribution?
• RQ3 To what extent does modeling the full joint distribution improve generalization and robustness in low-label-data regimes?
• RQ4 How does the flexibility of choosing different distribution families impact performance across diverse graph datasets?
• RQ5 Does the proposed framework maintain strong performance across both transductive and inductive learning settings?

Key findings

• The proposed framework achieves state-of-the-art performance on multiple benchmark graph-based semi-supervised learning datasets.
• The method consistently outperforms existing state-of-the-art models across most experimental settings, particularly in low-label-data regimes.
• The integration of features, labels, and graph structure through a joint generative model leads to more robust and accurate label predictions.
• Scalable variational inference enables effective and efficient posterior approximation, making the method practical for large-scale graphs.
• The flexibility of the framework in choosing different distribution families allows for adaptation to diverse data characteristics and graph types.
• Empirical results demonstrate that modeling the joint distribution yields significant improvements over methods that treat features, labels, and structure independently or with weak coupling.