[Paper Review] Semi-Supervised Learning on Graphs with Feature-Augmented Graph Basis Functions

Wolfgang Erb|arXiv (Cornell University)|Mar 17, 2020

Advanced Graph Neural Networks34 references4 citations

TL;DR

This paper proposes a semi-supervised learning framework on graphs using feature-augmented graph basis functions (GBFs) to improve classification accuracy by integrating geometric structure and additional feature information via Schur-Hadamard product updates. The method enhances standard GBF kernels with priors from unsupervised clustering or node attributes, achieving superior performance with fewer labeled nodes, especially on real-world datasets like Wisconsin Breast Cancer and Ionosphere.

ABSTRACT

For semi-supervised learning on graphs, we study how initial kernels in a supervised learning regime can be augmented with additional information from known priors or from unsupervised learning outputs. These augmented kernels are constructed in a simple update scheme based on the Schur-Hadamard product of the kernel with additional feature kernels. As generators of the positive definite kernels we will focus on graph basis functions (GBF) that allow to include geometric information of the graph via the graph Fourier transform. Using a regularized least squares (RLS) approach for machine learning, we will test the derived augmented kernels for the classification of data on graphs.

Motivation & Objective

• To improve semi-supervised learning on graphs by incorporating geometric structure and external priors into kernel functions.
• To develop a simple, efficient kernel augmentation strategy using the Schur-Hadamard product to combine graph basis functions with feature information.
• To demonstrate that feature-augmented GBF kernels outperform standard supervised GBF-RLS, especially with limited labeled data.
• To validate the approach on synthetic and real-world graph datasets using unsupervised priors derived from spectral clustering.

Proposed method

• Constructs positive definite graph basis functions (GBFs) using the graph Fourier transform to encode geometric structure.
• Generates feature kernels from auxiliary feature maps or graphs, such as those from spectral clustering.
• Forms a tensor-product kernel on the Cartesian product of the graph and feature graphs.
• Extracts a feature-augmented kernel via embedding into the product graph, expressed as a Schur-Hadamard product of the original GBF kernel and a feature matrix.
• Solves the classification task using regularized least squares (RLS) with the augmented kernel to enforce smoothness in the reproducing kernel Hilbert space.
• Employs normalized graph Laplacian for similarity graphs and applies Shi-Malik spectral clustering to generate binary feature graphs as priors.

Experimental results

Research questions

• RQ1 Can kernel-based semi-supervised learning on graphs be improved by integrating geometric and feature-based priors through kernel augmentation?
• RQ2 How does the Schur-Hadamard product of a GBF kernel with a feature kernel affect classification accuracy with limited labeled data?
• RQ3 To what extent do unsupervised clustering outputs (e.g., spectral clustering) serve as effective priors in semi-supervised graph learning?
• RQ4 Does feature augmentation reduce the required number of labeled nodes for achieving high classification accuracy?

Key findings

• On the Ionosphere dataset, the feature-augmented GBF-RLS achieved 90.05% accuracy with 60 labeled nodes, outperforming the supervised GBF-RLS (87.18%) and the unsupervised spectral classifier (71.79%).
• For the Wisconsin Breast Cancer dataset, the semi-supervised $oldsymbol{ ho}$-GBF-RLS reached 96.05% accuracy with 64 labeled nodes, matching the unsupervised spectral classifier, while supervised GBF-RLS achieved only 96.43% with full labeling.
• With only 2 labeled nodes, the feature-augmented method achieved 91.68% accuracy on WBC, compared to 72.73% for supervised GBF-RLS, demonstrating strong data efficiency.
• On the two-moon synthetic dataset, the feature-augmented kernel reduced the number of required labeled nodes by up to 50% to achieve comparable accuracy to the supervised baseline.
• The method maintains high performance even with large graphs, showing scalability and robustness to noise in feature priors.