[Paper Review] Sheaf Neural Networks with Connection Laplacians

Federico Barbero, Cristian Bodnar+4 more|arXiv (Cornell University)|Jun 17, 2022

Neural Networks and Applications4 citations

TL;DR

This paper proposes a deterministic, pre-processing method for computing sheaf Laplacians in Sheaf Neural Networks (SNNs) using orthogonal maps that optimally align tangent spaces of neighboring nodes under the manifold assumption. By leveraging differential geometry and graph structure, the method achieves competitive performance with reduced computational overhead and faster inference than gradient-based sheaf learning.

ABSTRACT

A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces. SNNs have been shown to have useful theoretical properties that help tackle issues arising from heterophily and over-smoothing. One complication intrinsic to these models is finding a good sheaf for the task to be solved. Previous works proposed two diametrically opposed approaches: manually constructing the sheaf based on domain knowledge and learning the sheaf end-to-end using gradient-based methods. However, domain knowledge is often insufficient, while learning a sheaf could lead to overfitting and significant computational overhead. In this work, we propose a novel way of computing sheaves drawing inspiration from Riemannian geometry: we leverage the manifold assumption to compute manifold-and-graph-aware orthogonal maps, which optimally align the tangent spaces of neighbouring data points. We show that this approach achieves promising results with less computational overhead when compared to previous SNN models. Overall, this work provides an interesting connection between algebraic topology and differential geometry, and we hope that it will spark future research in this direction.

Motivation & Objective

• Address the challenge of sheaf initialization in Sheaf Neural Networks (SNNs), where manual construction lacks generalization and end-to-end learning incurs high computational cost and overfitting risk.
• Overcome limitations of existing SNNs by introducing a non-parametric, pre-computed sheaf structure that embeds geometric inductive bias from data.
• Reduce training time and parameter count by eliminating the need to backpropagate through sheaf parameters during model optimization.
• Explore the integration of differential geometry (tangent space alignment) with graph-structured data to improve generalization in heterophilic settings.
• Demonstrate that a deterministic, manifold-aware sheaf computation can match or exceed performance of learnable sheaf models on diverse benchmark datasets.

Proposed method

• Compute local tangent spaces at each node using principal component analysis (PCA) on its 1-hop neighborhood to estimate the local manifold structure.
• Derive orthogonal transformation matrices $ O_{ij} $ between neighboring nodes that optimally align their respective tangent spaces, minimizing the distance between orthonormal bases.
• Construct a connection Laplacian from these orthogonal maps, forming a graph- and manifold-aware sheaf Laplacian without learnable parameters.
• Pre-compute the sheaf Laplacian during data preprocessing, decoupling it from the main model training phase.
• Integrate the pre-computed Laplacian into a Sheaf Neural Network (SNN) for inference and training, using standard GNN message-passing with the new sheaf structure.
• Ensure the method remains scalable and efficient by avoiding backpropagation through the sheaf, reducing memory and compute demands.

Experimental results

Research questions

• RQ1 Can a deterministic, geometry-driven method for computing sheaf Laplacians outperform end-to-end learned sheaves in terms of accuracy and efficiency?
• RQ2 Does incorporating differential geometric principles—specifically tangent space alignment—improve generalization in heterophilic graph learning scenarios?
• RQ3 To what extent can pre-computing the sheaf Laplacian reduce training time and parameter count without sacrificing model performance?
• RQ4 How does the proposed method compare to random orthogonal sheaf baselines and other SNN variants across datasets with varying homophily levels?
• RQ5 In which graph regimes (e.g., low-node, high-edge, low-homophily) does the deterministic sheaf computation provide the most significant advantage?

Key findings

• Conn-NSD achieves state-of-the-art performance on heterophilic datasets such as Texas and Film, outperforming both Diag-NSD, O(d)-NSD, and Gen-NSD.
• On the Squirrel dataset, which has high node and edge counts and poor MLP performance, Conn-NSD still outperforms random sheaf baselines, though it lags behind fully learnable models.
• Conn-NSD achieves a 45.8% speedup in inference time on the Squirrel dataset compared to the O(d)-NSD baseline, with faster training due to pre-computed Laplacian.
• The model consistently reduces computational overhead by eliminating gradient computation through the sheaf, with mean inference times of 0.010–0.310 seconds per epoch across datasets.
• Conn-NSD maintains competitive performance on Cora, Citeseer, and Pubmed, remaining close to the best-performing models while using fewer learnable parameters.
• The method demonstrates that geometric structure derived from manifold assumptions can serve as an effective inductive bias, reducing overfitting and improving generalization in SNNs.