[Paper Review] Label Efficient Semi-Supervised Learning via Graph Filtering

Qimai Li, Xiao-Ming Wu+3 more|arXiv (Cornell University)|Jan 28, 2019

Advanced Graph Neural Networks72 references20 citations

TL;DR

This paper proposes a graph filtering framework for label-efficient semi-supervised learning that unifies label propagation and graph convolutional networks (GCNs) by treating data features as signals on a graph and applying low-pass graph filters to smooth and enhance representations. The method achieves state-of-the-art performance with significantly fewer parameters and faster inference, especially under low-label regimes, through generalized label propagation (GLP) and an improved GCN (IGCN) with adjustable filter strength.

ABSTRACT

Graph-based methods have been demonstrated as one of the most effective approaches for semi-supervised learning, as they can exploit the connectivity patterns between labeled and unlabeled data samples to improve learning performance. However, existing graph-based methods either are limited in their ability to jointly model graph structures and data features, such as the classical label propagation methods, or require a considerable amount of labeled data for training and validation due to high model complexity, such as the recent neural-network-based methods. In this paper, we address label efficient semi-supervised learning from a graph filtering perspective. Specifically, we propose a graph filtering framework that injects graph similarity into data features by taking them as signals on the graph and applying a low-pass graph filter to extract useful data representations for classification, where label efficiency can be achieved by conveniently adjusting the strength of the graph filter. Interestingly, this framework unifies two seemingly very different methods -- label propagation and graph convolutional networks. Revisiting them under the graph filtering framework leads to new insights that improve their modeling capabilities and reduce model complexity. Experiments on various semi-supervised classification tasks on four citation networks and one knowledge graph and one semi-supervised regression task for zero-shot image recognition validate our findings and proposals.

Motivation & Objective

• To address the limitation of existing graph-based semi-supervised learning methods that either ignore data features (e.g., classical label propagation) or require large labeled datasets due to high model complexity (e.g., deep GCNs).
• To develop a label-efficient learning framework that leverages both graph structure and data features by modeling them as signals on a graph and applying low-pass filtering.
• To unify classical label propagation and modern GCNs under a single graph filtering perspective, enabling new insights and improved model design.
• To reduce model complexity and prevent overfitting in low-label settings through parameter-efficient filter design and flexible filter strength control.

Proposed method

• Treats data features as signals on a graph and applies a low-pass graph filter to smooth and extract representative features for classification.
• Proposes generalized label propagation (GLP) by extending label propagation with data feature matrices as input signals, arbitrary low-pass graph filters, and any classifier.
• Reinterprets GCN as performing Laplacian smoothing via a first-order approximation of spectral filtering, revealing the role of the renormalization trick and parameter settings.
• Introduces an improved GCN (IGCN) with an exponent parameter on the filter to control strength, reducing trainable parameters and improving generalization under label scarcity.
• Uses graph filtering to inject structural similarity into data features, enabling effective representation learning with minimal labeled data.
• Employs a flexible framework where filter strength can be tuned to balance inductive bias and model capacity, enhancing label efficiency.

Experimental results

Research questions

• RQ1 How can graph-based semi-supervised learning be unified under a single theoretical framework that integrates both graph structure and data features?
• RQ2 Can classical label propagation and modern GCNs be reinterpreted as instances of graph filtering, and what new insights does this provide?
• RQ3 How can filter strength be controlled to achieve label efficiency in low-supervision regimes?
• RQ4 Can a unified framework improve performance while reducing model complexity and overfitting in data-scarce settings?

Key findings

• The proposed graph filtering framework unifies label propagation and GCNs, revealing that both can be interpreted as low-pass filtering of data signals on a graph.
• Generalized label propagation (GLP) achieves state-of-the-art performance on citation networks and zero-shot image recognition, outperforming methods like GCNZ and DGPM.
• IGCN with a tunable exponent parameter reduces trainable parameters and prevents overfitting, improving performance on limited-label datasets.
• Replacing a 6-layer GCN in GCNZ with a 2-layer IGCN and GLP boosts accuracy from 68% to around 73%, demonstrating superior efficiency and performance.
• GLP runs hundreds of times faster than GCNZ and tens of times faster than other baselines, highlighting its computational efficiency.
• Smaller filter exponents (e.g., k=2) yield better performance on zero-shot regression tasks, suggesting that preserving feature diversity is critical.