[Paper Review] Graph Structure Learning for Robust Graph Neural Networks

Wei Jin, Yao Ma+4 more|arXiv (Cornell University)|May 20, 2020

Advanced Graph Neural NetworksComputer Science41 references49 citations

TL;DR

Pro-GNN jointly learns a clean graph structure and robust GNN parameters from perturbed graphs by enforcing low-rank, sparsity, and feature smoothness, yielding strong defense against various graph adversarial attacks.

ABSTRACT

Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, recent studies show that GNNs are vulnerable to carefully-crafted perturbations, called adversarial attacks. Adversarial attacks can easily fool GNNs in making predictions for downstream tasks. The vulnerability to adversarial attacks has raised increasing concerns for applying GNNs in safety-critical applications. Therefore, developing robust algorithms to defend adversarial attacks is of great significance. A natural idea to defend adversarial attacks is to clean the perturbed graph. It is evident that real-world graphs share some intrinsic properties. For example, many real-world graphs are low-rank and sparse, and the features of two adjacent nodes tend to be similar. In fact, we find that adversarial attacks are likely to violate these graph properties. Therefore, in this paper, we explore these properties to defend adversarial attacks on graphs. In particular, we propose a general framework Pro-GNN, which can jointly learn a structural graph and a robust graph neural network model from the perturbed graph guided by these properties. Extensive experiments on real-world graphs demonstrate that the proposed framework achieves significantly better performance compared with the state-of-the-art defense methods, even when the graph is heavily perturbed. We release the implementation of Pro-GNN to our DeepRobust repository for adversarial attacks and defenses (footnote: https://github.com/DSE-MSU/DeepRobust). The specific experimental settings to reproduce our results can be found in https://github.com/ChandlerBang/Pro-GNN.

Motivation & Objective

• Motivation: Graph Neural Networks are vulnerable to adversarial perturbations of graph structure.
• Goal: Develop a defense that cleans the perturbed graph while training a robust GNN.
• Approach: Leverage intrinsic graph properties (low rank, sparsity, feature smoothness) to learn a clean adjacency while optimizing GNN parameters.
• Contribution: A unified framework (Pro-GNN) that jointly learns the clean graph and robust GNN under poisoning attacks.

Proposed method

• Learn a clean adjacency S from the poisoned A by promoting low rank and sparsity via nuclear and L1 norms (|S|* and |S|1) while keeping S close to A and symmetric.
• Incorporate a feature-smoothness term Ls = tr(X^T L_hat X) to encourage similar features on connected nodes, with L_hat the normalized Laplacian of S.
• Formulate a joint objective: L = ||A-S||_F^2 + alpha||S||_1 + beta||S||_* + gamma L_GNN + lambda tr(X^T L_hat X).
• Use alternating optimization to update GNN parameters theta (with fixed S) and the graph S (with proximal steps for the non-differentiable norms).
• Apply proximal operators for L1 and nuclear norms and project S onto the feasible set (symmetry, [0,1] entries).
• Provide an end-to-end training algorithm (Algorithm 1) that iteratively updates S and theta.

Experimental results

Research questions

• RQ1 Can Pro-GNN defend against multiple types of graph adversarial attacks (non-targeted, targeted, random)?
• RQ2 Does jointly learning the graph and GNN improve robustness compared to two-stage graph cleaning followed by training?
• RQ3 How do low-rank, sparsity, and feature-smoothness constraints impact defense performance?
• RQ4 Is the learned graph S useful for improving node classification on unlabeled nodes under perturbations?

Key findings

• Pro-GNN achieves significantly better defense performance than state-of-the-art methods across several real-world graphs under heavy perturbations.
• Jointly learning the clean graph and GNN yields better results than two-stage preprocessing approaches.
• The framework leverages low-rank, sparsity, and feature-smoothness properties to suppress adversarial structure while preserving task-relevant connectivity.
• Experiments on Cora, Citeseer, PubMed, and Polblogs demonstrate robustness against non-targeted and targeted attacks, outperforming baselines like GCN, GAT, RGCN, GCN-Jaccard, and GCN-SVD.
• A variant without the feature-smoothness term (Pro-GNN-fs) confirms the importance of Ls in defense effectiveness.