[Paper Review] Uncertainty Aware Semi-Supervised Learning on Graph Data

Xujiang Zhao, Feng Chen+2 more|arXiv (Cornell University)|Oct 24, 2020

Advanced Graph Neural Networks28 references53 citations

TL;DR

The paper introduces a multi-source uncertainty framework for GNNs that models multinomial opinions via node-level Dirichlet distributions, uses GKDE for improved uncertainty estimation, and employs a teacher network for enhanced semi-supervised node classification and OOD detection.

ABSTRACT

Thanks to graph neural networks (GNNs), semi-supervised node classification has shown the state-of-the-art performance in graph data. However, GNNs have not considered different types of uncertainties associated with class probabilities to minimize risk of increasing misclassification under uncertainty in real life. In this work, we propose a multi-source uncertainty framework using a GNN that reflects various types of predictive uncertainties in both deep learning and belief/evidence theory domains for node classification predictions. By collecting evidence from the given labels of training nodes, the Graph-based Kernel Dirichlet distribution Estimation (GKDE) method is designed for accurately predicting node-level Dirichlet distributions and detecting out-of-distribution (OOD) nodes. We validated the outperformance of our proposed model compared to the state-of-the-art counterparts in terms of misclassification detection and OOD detection based on six real network datasets. We found that dissonance-based detection yielded the best results on misclassification detection while vacuity-based detection was the best for OOD detection. To clarify the reasons behind the results, we provided the theoretical proof that explains the relationships between different types of uncertainties considered in this work.

Motivation & Objective

• Motivate incorporating multiple uncertainty types into graph-based semi-supervised learning to reduce misclassification risk under real-world uncertainty.
• Develop a framework that combines deep learning and belief/evidence theory to estimate diverse predictive uncertainties at the node level.
• Enable accurate detection of misclassifications and out-of-distribution (OOD) nodes in graph data.
• Provide theoretical insights into the relationships between different uncertainty types in graph-based predictions.

Proposed method

• Represent node predictions as multinomial opinions equivalently modeled by Dirichlet distributions with node-specific parameters alpha_i = f_i(A, r; theta).
• Propose a Subjective Bayesian GNN (S-BGNN) that integrates dropout-based Bayesian inference to approximate P(y|A, r; G) and learn a distribution over model parameters.
• Introduce GKDE (Graph-based Kernel Dirichlet distribution Estimation) to estimate prior Dirichlet parameters for nodes using graph structure and training labels, guiding uncertainty estimation.
• Incorporate a Teacher Network via iterative knowledge distillation to regularize and refine node probability outputs, minimizing KL divergences between Dirichlet distributions and teacher predictions.
• Provide a loss function that combines squared error with variance to train the model and reduce bias while controlling uncertainty.
• Leverage theoretical results (e.g., relations among vacuity, dissonance, entropy, etc.) to interpret when each uncertainty type is informative for OOD vs misclassification detection.

Experimental results

Research questions

• RQ1 RQ1: Which type of predictive uncertainty most effectively signals misclassification in semi-supervised node classification on graphs?
• RQ2 RQ2: Which uncertainty type best indicates out-of-distribution (OOD) nodes in graph data?
• RQ3 RQ3: Can GKDE improve the estimation of node-level Dirichlet distributions and enhance uncertainty-aware predictions?
• RQ4 RQ4: Does incorporating a Teacher Network and GKDE materially improve classification accuracy and uncertainty estimates?

Key findings

• Dissonance-based detection most effectively identifies misclassifications among evaluated uncertainty types.
• Vacuity-based detection provides the strongest signal for detecting OOD nodes.
• GKDE improves the quality of node-level Dirichlet distributions and enhances OOD detection performance.
• A Teacher Network further boosts node classification accuracy and uncertainty estimation in the proposed framework.