[论文解读] Unifying over-smoothing and over-squashing in graph neural networks: A physics informed approach and beyond

Zhiqi Shao, Dai Shi+4 more|arXiv (Cornell University)|Sep 6, 2023

Advanced Graph Neural Networks被引用 4

一句话总结

本文提出了一种物理启发的图神经网络 MHKG 及其泛化形式 G-MHKG，通过反转图热方程的时间方向以生成高通滤波效果，统一缓解了过平滑与过压缩问题。该方法利用多尺度热核滤波函数平衡节点特征的平滑与锐化，揭示了过平滑与过压缩之间的根本权衡关系，且 G-MHKG 在同质图与异质图上均取得了当前最优性能。

ABSTRACT

Graph Neural Networks (GNNs) have emerged as one of the leading approaches for machine learning on graph-structured data. Despite their great success, critical computational challenges such as over-smoothing, over-squashing, and limited expressive power continue to impact the performance of GNNs. In this study, inspired from the time-reversal principle commonly utilized in classical and quantum physics, we reverse the time direction of the graph heat equation. The resulted reversing process yields a class of high pass filtering functions that enhance the sharpness of graph node features. Leveraging this concept, we introduce the Multi-Scaled Heat Kernel based GNN (MHKG) by amalgamating diverse filtering functions' effects on node features. To explore more flexible filtering conditions, we further generalize MHKG into a model termed G-MHKG and thoroughly show the roles of each element in controlling over-smoothing, over-squashing and expressive power. Notably, we illustrate that all aforementioned issues can be characterized and analyzed via the properties of the filtering functions, and uncover a trade-off between over-smoothing and over-squashing: enhancing node feature sharpness will make model suffer more from over-squashing, and vice versa. Furthermore, we manipulate the time again to show how G-MHKG can handle both two issues under mild conditions. Our conclusive experiments highlight the effectiveness of proposed models. It surpasses several GNN baseline models in performance across graph datasets characterized by both homophily and heterophily.

研究动机与目标

• 解决 GNN 中长期存在的过平滑、过压缩及表达能力有限的统一处理难题。
• 通过借鉴物理学中的时间反演，探究过平滑与过压缩之间的内在关联。
• 构建一种灵活且可分析的 GNN 框架，实现对特征平滑、拓扑压缩与表达能力的系统性控制。
• 证明过平滑与过压缩在谱域中可通过滤波函数特性实现根本关联。

提出的方法

• 反转图热方程的时间方向，生成时间反演过程，作为高通滤波器增强特征锐度。
• 通过结合来自热核与反向热核的低通（平滑）与高通（锐化）滤波函数，构建多尺度热核 GNN（MHKG）。
• 将 MHKG 泛化为 G-MHKG，以支持灵活的滤波函数设计与模型组件控制，包括可学习参数与谱滤波矩阵。
• 利用图拉普拉斯矩阵的谱分解，将滤波操作表示在特征基下，实现对能量动态与特征传播的解析控制。
• 在 G-MHKG 中引入时间操控策略，以自适应处理过平滑与过压缩问题，尤其在异质性图中表现优异。
• 利用图框架的紧致性原理，将 G-MHKG-F 解释为非紧致变体，从而在多种图类型上实现鲁棒性能。

Figure 1 : Top: The evolution of the node feature of diffusion (smoothing) process (i.e., from distinct features to over-smoothing). Bottom: the reverse diffusion (sharpening) process (i.e., from nearly identical to distinct node features).

实验结果

研究问题

• RQ1 能否基于物理学原理，将 GNN 中的过平滑与过压缩统一于单一理论框架下？
• RQ2 过平滑与过压缩之间是否存在内在关联？能否通过滤波函数特性加以表征？
• RQ3 单一 GNN 架构能否同时有效缓解过平滑与过压缩？在何种条件下可实现？
• RQ4 不同滤波函数（如正弦、指数）在同质图与异质图上的性能表现有何差异？
• RQ5 在实际图学习任务中，基于框架的滤波紧致性在多大程度上具有实际意义？

主要发现

• 所提出的 G-MHKG 模型在同质图（Cora、Citeseer、PubMed）与异质图（Cornell、Texas、Wisconsin）基准上均达到当前最优性能。
• G-MHKG-F 在 Cornell 上达到 88.2% 准确率，Texas 上为 86.1%，Wisconsin 上为 84.7%，优于 GAT、GIN 与 UFG 等基线模型。
• 揭示了过平滑与过压缩之间存在根本性权衡：通过高通滤波增强特征锐度会加剧过压缩，反之亦然。
• G-MHKG 中的时间操控策略能有效应对异质性图中的两类问题，但在同质性图中理论上无法同时完全解决两者。
• MHKG-I 变体表现欠佳，原因在于负特征值缩放导致滤波矩阵非正定，凸显了合理滤波函数设计的重要性。
• G-MHKG-F 在异质性图上表现可与或优于紧致图框架（UFG），表明紧致性在实际性能中可能并非必要条件。

Figure 2 : The figure on the left represents different types of HFD filtering outcomes and the trade-off between two issues. One can check that to induce more sharpening (filtering function from bottom to top), the model will suffer more from OSQ. The figure on the right illustrates the situation de