[论文解读] Understanding convolution on graphs via energies

Francesco Di Giovanni, James Rowbottom+3 more|arXiv (Cornell University)|Jun 22, 2022

Advanced Graph Neural Networks被引用 12

一句话总结

本文表明，通过展示其作为最小化广义能量函数的梯度流，标准图卷积可增强高频特征而非始终平滑化，该函数通过对称权重矩阵实现吸引与排斥。关键贡献在于证明此类模型可避免过度平滑，甚至表现出过度锐化，通过多粒子能量最小化为GNN动力学提供了新的物理解释。

ABSTRACT

Graph Neural Networks (GNNs) typically operate by message-passing, where the state of a node is updated based on the information received from its neighbours. Most message-passing models act as graph convolutions, where features are mixed by a shared, linear transformation before being propagated over the edges. On node-classification tasks, graph convolutions have been shown to suffer from two limitations: poor performance on heterophilic graphs, and over-smoothing. It is common belief that both phenomena occur because such models behave as low-pass filters, meaning that the Dirichlet energy of the features decreases along the layers incurring a smoothing effect that ultimately makes features no longer distinguishable. In this work, we rigorously prove that simple graph-convolutional models can actually enhance high frequencies and even lead to an asymptotic behaviour we refer to as over-sharpening, opposite to over-smoothing. We do so by showing that linear graph convolutions with symmetric weights minimize a multi-particle energy that generalizes the Dirichlet energy; in this setting, the weight matrices induce edge-wise attraction (repulsion) through their positive (negative) eigenvalues, thereby controlling whether the features are being smoothed or sharpened. We also extend the analysis to non-linear GNNs, and demonstrate that some existing time-continuous GNNs are instead always dominated by the low frequencies. Finally, we validate our theoretical findings through ablations and real-world experiments.

研究动机与目标

• 探究标准图卷积是否本质上平滑特征，或可增强高频分量。
• 挑战图卷积始终作为低通滤波器、导致深层网络中过度平滑的普遍观点。
• 将图卷积物理解释为受能量函数支配的多粒子动力学，通过权重矩阵的特征值实现吸引与排斥。
• 从能量最小化和频率响应角度分析线性和非线性GNN的渐近行为。
• 通过消融研究和在异配性与同配性图上的真实世界实验验证理论发现。

提出的方法

• 提出广义能量泛函 $\mathcal{E}_{\theta}$，其推广了Dirichlet能量，可捕捉图卷积的动力学行为。
• 表明具有对称权重矩阵的线性图卷积是梯度流，可最小化 $\mathcal{E}_{\theta}$，从而将模型动力学与能量最小化联系起来。
• 分析权重矩阵特征值的作用：正特征值引发吸引（平滑），负特征值引发排斥（锐化）。
• 将分析扩展至非线性GNN，表明时间连续GNN受低频主导，与所提出的梯度流模型不同。
• 提出GRAFF与GRAFF$_{NL}$模型，基于对称权重矩阵与残差连接，验证其在异配性图上的性能提升。
• 采用谱分析与真实世界数据集上的消融研究，验证能量衰减与特征增强的动力学。

实验结果

研究问题

• RQ1 标准图卷积是否可增强高频分量而非始终平滑特征？
• RQ2 图卷积是否始终作为低通滤波器，或在深层架构中可表现出过度锐化？
• RQ3 Dirichlet能量是否足以描述GNN行为，还是更一般的能量泛函更能解释特征演化？
• RQ4 图卷积中的对称权重矩阵是否可诱导排斥力，从而提升异配性图上的性能？
• RQ5 非线性与时间连续GNN在频率响应与能量动力学方面如何比较？

主要发现

• 具有对称权重的线性图卷积可最小化广义能量 $\mathcal{E}_{\theta}$，当存在负特征值时可导致特征锐化。
• 理论分析证明此类模型可增强高频分量并避免过度平滑，其渐近行为称为“过度锐化”。
• 实证结果表明，GRAFF与GRAFF$_{NL}$在Chameleon与Squirrel等异配性数据集上优于基线模型，分别取得68.99%与52.48%的测试准确率。
• 在Cora数据集上，模型取得87.4%的测试准确率，Dirichlet能量仅增加0.11，表明其具有非平滑动力学。
• 谱分析证实，GRAFF中学习到的权重矩阵具有支持吸引与排斥的特征值分布，从而实现高频分量的保留。
• 本研究证明，只要权重矩阵对称并诱导能量最小化动力学，深层GNN中的过度平滑并非不可避免。