[论文解读] Graph Structure Learning for Robust Graph Neural Networks

Wei Jin, Yao Ma+4 more|arXiv (Cornell University)|May 20, 2020

Advanced Graph Neural Networks参考文献 41被引用 49

一句话总结

Pro-GNN 通过在受扰动图上强制低秩、稀疏性和特征光滑性，联合学习干净的图结构和鲁棒的 GNN 参数，从而对各种图对抗攻击具有强鲁棒性。

ABSTRACT

Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, recent studies show that GNNs are vulnerable to carefully-crafted perturbations, called adversarial attacks. Adversarial attacks can easily fool GNNs in making predictions for downstream tasks. The vulnerability to adversarial attacks has raised increasing concerns for applying GNNs in safety-critical applications. Therefore, developing robust algorithms to defend adversarial attacks is of great significance. A natural idea to defend adversarial attacks is to clean the perturbed graph. It is evident that real-world graphs share some intrinsic properties. For example, many real-world graphs are low-rank and sparse, and the features of two adjacent nodes tend to be similar. In fact, we find that adversarial attacks are likely to violate these graph properties. Therefore, in this paper, we explore these properties to defend adversarial attacks on graphs. In particular, we propose a general framework Pro-GNN, which can jointly learn a structural graph and a robust graph neural network model from the perturbed graph guided by these properties. Extensive experiments on real-world graphs demonstrate that the proposed framework achieves significantly better performance compared with the state-of-the-art defense methods, even when the graph is heavily perturbed. We release the implementation of Pro-GNN to our DeepRobust repository for adversarial attacks and defenses (footnote: https://github.com/DSE-MSU/DeepRobust). The specific experimental settings to reproduce our results can be found in https://github.com/ChandlerBang/Pro-GNN.

研究动机与目标

• 动机：图神经网络容易受到图结构的对抗性扰动影响。
• 目标：在训练鲁棒 GNN 的同时，清理受扰动的图。
• 方法：利用图的内在属性（低秩、稀疏性、特征光滑性）在优化 GNN 参数的同时学习一个干净的邻接矩阵。
• 贡献：一个统一框架（Pro-GNN），在投毒攻击下同时学习干净图和鲁棒的 GNN。

提出的方法

• 通过对|S|*（核范数）和|S|1（L1 范数）促进低秩和稀疏性，同时保持 S 接近 A 且对称，学习干净的邻接矩阵 S。
• 引入特征光滑项 Ls = tr(X^T L_hat X)，以在连接的节点之间鼓励相似的特征，其中 L_hat 是 S 的归一化拉普拉斯矩阵。
• 给出联合目标：L = ||A-S||_F^2 + alpha||S||_1 + beta||S||_* + gamma L_GNN + lambda tr(X^T L_hat X)。
• 使用交替优化来更新 GNN 参数 theta（在固定 S 时）以及图 S（对不可微的范数进行近端步）。
• 对 L1 和核范数应用近端算子，并将 S 投影到可行集合（对称、元素在 [0,1]）。
• 给出端到端的训练算法（算法 1），迭代更新 S 和 theta。

实验结果

研究问题

• RQ1 Pro-GNN 是否能对多种类型的图对抗攻击（非目标攻击、定向、随机等）进行防御？
• RQ2 与先进行两阶段的图清理再训练相比，联合学习图和 GNN 是否能提升鲁棒性？
• RQ3 低秩、稀疏性和特征光滑性约束如何影响防御性能？
• RQ4 在存在扰动的情况下，学习得到的图 S 是否有助于提升未标记节点的节点分类性能？

主要发现

• 在多张真实图上，在强扰动下，Pro-GNN 的防御性能显著优于最新方法。
• 联合学习干净图和 GNN 比两阶段预处理方法效果更好。
• 该框架利用低秩、稀疏性和特征光滑性属性，在抑制对抗结构的同时保留与任务相关的连通性。
• 在 Cora、Citeseer、PubMed 和 Polblogs 上的实验表明，对非目标和定向攻击具有鲁棒性，超越如 GCN、GAT、RGCN、GCN-Jaccard、GCN-SVD 等基线。
• 一个不含特征光滑项的变体（Pro-GNN-fs）证实了 Ls 在防御效果中的重要性。