[论文解读] Interpreting and Unifying Graph Neural Networks with An Optimization Framework

Meiqi Zhu, Xiao Wang+3 more|arXiv (Cornell University)|Jan 28, 2021

Advanced Graph Neural Networks参考文献 49被引用 28

一句话总结

本文提出一个统一的优化框架，将多种 GNN 传播机制解释为特征拟合加图拉普拉斯正则化目标的解，并在该框架下推导出具备低通和高通滤波器的新型 GNN。

ABSTRACT

Graph Neural Networks (GNNs) have received considerable attention on graph-structured data learning for a wide variety of tasks. The well-designed propagation mechanism which has been demonstrated effective is the most fundamental part of GNNs. Although most of GNNs basically follow a message passing manner, litter effort has been made to discover and analyze their essential relations. In this paper, we establish a surprising connection between different propagation mechanisms with a unified optimization problem, showing that despite the proliferation of various GNNs, in fact, their proposed propagation mechanisms are the optimal solution optimizing a feature fitting function over a wide class of graph kernels with a graph regularization term. Our proposed unified optimization framework, summarizing the commonalities between several of the most representative GNNs, not only provides a macroscopic view on surveying the relations between different GNNs, but also further opens up new opportunities for flexibly designing new GNNs. With the proposed framework, we discover that existing works usually utilize naive graph convolutional kernels for feature fitting function, and we further develop two novel objective functions considering adjustable graph kernels showing low-pass or high-pass filtering capabilities respectively. Moreover, we provide the convergence proofs and expressive power comparisons for the proposed models. Extensive experiments on benchmark datasets clearly show that the proposed GNNs not only outperform the state-of-the-art methods but also have good ability to alleviate over-smoothing, and further verify the feasibility for designing GNNs with our unified optimization framework.

研究动机与目标

• 倡导对多样化 GNN 传播机制的宏观、原理性视角。
• 提出一个统一的优化目标，将特征拟合与图正则化结合起来。
• 展示代表性的 GNN（GCN、SGC、PPNP/APPNP、JKNet、DAGNN）如何融入该框架。
• 设计两种具可调低通和高通图核的新型 GNN。
• 提供关于收敛性和表达能力的理论分析，以及经验验证。

提出的方法

• 定义一个由特征拟合项和图拉普拉斯正则化项组成的双项目标。
• 引入可灵活调整的图卷积核 F1 和 F2，以实现不同的滤波行为。
• 证明若干 GNN 可以视为该目标在特定 F1、F2 及正则化参数下的特例。
• 推导在统一目标下传播的闭式/迭代解。
• 开发两种新型 GNN 模型（GNN-LF 和 GNN-HF），具备低通和高通核，并分析它们的收敛性与表达能力。
• 提供将现有模型与统一框架连接起来的理论联系（定理），并讨论对设计的影响。

实验结果

研究问题

• RQ1 是否能够在一个具有特征拟合项和拉普拉斯正则化项的单一优化框架下，将常见的 GNN 传播机制统一起来？
• RQ2 不同的图卷积核（F1、F2）的选择如何对应已知的 GNN？是否能因此启发新的体系结构？
• RQ3 在该框架内的新颖低通和高通核是否能缓解过平滑并提升表达能力？
• RQ4 所提出的 GNN-LF 和 GNN-HF 模型的收敛性质与表达能力是什么？
• RQ5 该框架能否为未来 GNN 的可解释目标导向设计提供原则性指导？

主要发现

• 一个统一目标能够复现若干代表性 GNN 的传播机制，并通过双项优化将它们连接起来。
• GNN 如 SGC/GCN 对应于纯图正则化目标，而 APPNP/PPNP 引入了编码原始特征的拟合项。
• 该框架使得设计两个具可调低通与高通滤波核的新型 GNN 成为可能，能够缓解过平滑。
• 理论分析提供所提模型的收敛性和表达能力比较。
• 大量实验表明所提 GNN 在性能上优于最新方法，并验证在统一框架下设计 GNN 的可行性。