[Paper Review] On the Supermodularity of Active Graph-based Semi-supervised Learning with Stieltjes Matrix Regularization

Pin‐Yu Chen, Dennis Wei|arXiv (Cornell University)|Apr 9, 2018

Advanced Graph Neural Networks14 references3 citations

TL;DR

This paper proves the supermodularity of an active graph-based semi-supervised learning (AG-SSL) objective function under Stieltjes matrix regularization, enabling a greedy algorithm with theoretical performance guarantees. The method achieves perfect community detection with only 2 labeled nodes on real-world networks, outperforming state-of-the-art graph signal sampling methods under limited labeling budgets.

ABSTRACT

Active graph-based semi-supervised learning (AG-SSL) aims to select a small set of labeled examples and utilize their graph-based relation to other unlabeled examples to aid in machine learning tasks. It is also closely related to the sampling theory in graph signal processing. In this paper, we revisit the original formulation of graph-based SSL and prove the supermodularity of an AG-SSL objective function under a broad class of regularization functions parameterized by Stieltjes matrices. Under this setting, supermodularity yields a novel greedy label sampling algorithm with guaranteed performance relative to the optimal sampling set. Compared to three state-of-the-art graph signal sampling and recovery methods on two real-life community detection datasets, the proposed AG-SSL method attains superior classification accuracy given limited sample budgets.

Motivation & Objective

• To establish supermodularity of the AG-SSL objective under a broad class of Stieltjes matrix regularizers, including unnormalized and normalized graph Laplacians.
• To develop a greedy label sampling algorithm with performance guarantees relative to the optimal set, avoiding exhaustive search over all possible subsets.
• To improve classification accuracy in semi-supervised learning with minimal labeling effort, particularly in community detection tasks.
• To demonstrate superior performance compared to existing graph signal sampling and recovery methods under constrained sample budgets.

Proposed method

• Formulates graph-based SSL as a signal recovery problem with a multivariate Gaussian prior on labels, using a precision matrix derived from Stieltjes matrices.
• Proves supermodularity of the objective function for regularization matrices that are positive semidefinite with non-positive off-diagonal entries, such as graph Laplacians.
• Proposes a greedy algorithm that iteratively selects nodes to minimize expected prediction loss, leveraging the supermodularity for bounded performance.
• Uses the graph Fourier transform and spectral properties to define the signal model, without assuming bandlimitedness.
• Employs a loss function based on the expected posterior variance of the label signal, optimized via node selection.
• Implements a predictor based on ridge regression with Stieltjes matrix regularization, using noisy observations of labels.

Experimental results

Research questions

• RQ1 Does the AG-SSL objective function exhibit supermodularity under Stieltjes matrix regularization?
• RQ2 Can a greedy algorithm achieve a constant-factor approximation to the optimal sampling set under this supermodularity?
• RQ3 How does the proposed method compare to state-of-the-art graph signal sampling methods in terms of classification accuracy under limited labeling budgets?
• RQ4 What is the impact of using different regularization matrices (e.g., L vs. LN) on sampling performance and accuracy?

Key findings

• The proposed AG-SSL method achieves 100% classification accuracy in community detection using only 2 labeled nodes on both the Karate and Dolphin networks.
• The method outperforms Random sampling, Graph Spectral Proxy (GSP), Graph Shift Operator (GSO), and Chamon-Ribeiro’s method (CRM) in terms of both loss and accuracy under limited sample budgets.
• Using the normalized Laplacian (LN) instead of the unnormalized Laplacian (L) improves the performance of GSP and CRM, indicating the influence of the Graph Fourier basis on sampling quality.
• The proposed method shows significant overlap with GSP in node selection, suggesting alignment with spectral-based sampling heuristics, while GSO and CRM exhibit more divergent sampling patterns.
• Theoretical guarantees from supermodularity ensure that the greedy algorithm’s performance is bounded within a constant factor of the optimal sampling set, despite the NP-hardness of the full search.