[Paper Review] NAFS: A Simple yet Tough-to-beat Baseline for Graph Representation Learning

Wentao Zhang, Zeang Sheng+5 more|arXiv (Cornell University)|Jun 17, 2022

Advanced Graph Neural Networks7 citations

TL;DR

NAFS is a training-free, non-parametric graph representation learning method that enhances node embeddings through node-adaptive feature smoothing and feature ensemble across multiple neighborhood hops. It outperforms state-of-the-art GNNs on node clustering and link prediction while achieving up to 88.6× faster training, effectively mitigating over-smoothing and scalability issues in deep GNNs.

ABSTRACT

Recently, graph neural networks (GNNs) have shown prominent performance in graph representation learning by leveraging knowledge from both graph structure and node features. However, most of them have two major limitations. First, GNNs can learn higher-order structural information by stacking more layers but can not deal with large depth due to the over-smoothing issue. Second, it is not easy to apply these methods on large graphs due to the expensive computation cost and high memory usage. In this paper, we present node-adaptive feature smoothing (NAFS), a simple non-parametric method that constructs node representations without parameter learning. NAFS first extracts the features of each node with its neighbors of different hops by feature smoothing, and then adaptively combines the smoothed features. Besides, the constructed node representation can further be enhanced by the ensemble of smoothed features extracted via different smoothing strategies. We conduct experiments on four benchmark datasets on two different application scenarios: node clustering and link prediction. Remarkably, NAFS with feature ensemble outperforms the state-of-the-art GNNs on these tasks and mitigates the aforementioned two limitations of most learning-based GNN counterparts.

Motivation & Objective

• To address the over-smoothing and scalability limitations of deep GNNs in graph representation learning.
• To explore whether simple, non-parametric feature smoothing can outperform complex, trainable GNN architectures.
• To develop a scalable, efficient baseline that integrates structural and feature information without training.
• To provide a new, interpretable perspective on effective graph representation learning beyond end-to-end GNNs.

Proposed method

• NAFS performs feature smoothing by aggregating node features from neighbors at increasing hops using powers of the normalized adjacency matrix.
• It applies node-adaptive weighting to combine smoothed features across different hops, dynamically adjusting based on each node’s local structural properties.
• The method uses feature ensemble—via mean, max, or concatenation—across multiple smoothing strategies to enrich representation diversity.
• It avoids parameter learning entirely, making it training-free and highly efficient for large-scale graphs.
• The final node embedding is formed by adaptively combining multi-hop smoothed features, reducing over-smoothing risk.
• It leverages the normalized adjacency matrix $\hat{\mathbf{A}} = \mathbf{D}^{-1/2}\mathbf{A}\mathbf{D}^{-1/2}$ for stable and scalable smoothing operations.

Experimental results

Research questions

• RQ1 Can a non-parametric, training-free feature smoothing method outperform state-of-the-art trainable GNNs in graph representation learning?
• RQ2 How does node-adaptive weighting of multi-hop smoothed features improve performance compared to uniform or single-hop strategies?
• RQ3 To what extent can feature ensemble across different smoothing operators enhance representation quality?
• RQ4 Does NAFS effectively mitigate the over-smoothing problem in deep graph representations?
• RQ5 How does NAFS scale compared to GNNs on large graphs with limited memory and compute?

Key findings

• NAFS outperforms GAE and AGE by 9.0% and 3.8% in NMI on PubMed for node clustering, respectively.
• NAFS achieves up to 65.4× and 88.6× training speedup over GAE and AGE on PubMed, respectively.
• On ogbn-products, NAFS scales to 100,000 nodes with 23.7s inference, while GAE and AGE fail to train beyond 30,000 nodes due to OOM errors.
• The adaptive weighting strategy in NAFS prevents over-smoothing at high smoothing steps, unlike naive averaging or single-hop methods.
• T-SNE visualizations confirm that NAFS maintains class separability even at 200 smoothing hops, while baseline $\hat{\mathbf{A}}^k\mathbf{X}$ becomes completely disordered.
• Feature ensemble (mean, max, concat) consistently improves performance across datasets, with no single strategy dominating universally.