[논문 리뷰] Deep Iterative and Adaptive Learning for Graph Neural Networks

Yu Chen, Lingfei Wu외 1명|arXiv (Cornell University)|2019. 12. 17.

Advanced Graph Neural Networks참고 문헌 36인용 수 42

한 줄 요약

{

ABSTRACT

In this paper, we propose an end-to-end graph learning framework, namely Deep Iterative and Adaptive Learning for Graph Neural Networks (DIAL-GNN), for jointly learning the graph structure and graph embeddings simultaneously. We first cast the graph structure learning problem as a similarity metric learning problem and leverage an adapted graph regularization for controlling smoothness, connectivity and sparsity of the generated graph. We further propose a novel iterative method for searching for a hidden graph structure that augments the initial graph structure. Our iterative method dynamically stops when the learned graph structure approaches close enough to the optimal graph. Our extensive experiments demonstrate that the proposed DIAL-GNN model can consistently outperform or match state-of-the-art baselines in terms of both downstream task performance and computational time. The proposed approach can cope with both transductive learning and inductive learning.

연구 동기 및 목표

• Motivate automatic graph structure learning when graphs are unavailable or noisy.
• Propose an end-to-end framework that jointly learns graph structure and GNN parameters.
• Cast graph learning as similarity metric learning with graph regularization to control smoothness and sparsity.
• Develop an iterative method that dynamically stops when the graph is near optimal for the task.

제안 방법

• Design a learnable multi-head cosine similarity to construct an initial symmetric similarity S between node features across m perspectives.
• Extract a sparse adjacency A by applying an epsilon-neighborhood mask on S to reduce unnecessary connections.
• Regularize the learned graph using a Dirichlet energy term to enforce smoothness plus a sparsity/connectivity term f(A) that discourages disconnected graphs.
• Form a joint loss L = L_pred + L_G that couples graph learning with the downstream prediction task, and use a two-layer GCN for embeddings.
• Combine the learned adjacency with the initial graph A0 via a balanced mix in A-tilde = lambda L0 + (1-lambda) A/row-sum(A) to preserve useful prior structure.
• Iteratively update embeddings Z from A-tilde and refine A using the embeddings, with dynamic stopping when convergence criteria are met (or max iterations reached).
• Optionally fuse raw feature-based and embedding-based graphs as A-bar = eta A-tilde + (1-eta) A-tilde(0) to leverage both sources.

실험 결과

연구 질문

• RQ1 Can a jointly learned graph structure improve GNN task performance compared to fixed graphs or separately learned graphs?
• RQ2 Is an iterative, dynamically stopping graph refinement approach robust to noisy or incomplete initial graphs?
• RQ3 How does combining initial graph structure with learned adjustments impact transductive and inductive learning settings?
• RQ4 What is the computational trade-off of iterative graph learning versus fixed-graph GNN baselines?
• RQ5 How do the learned graphs fare under edge deletions or insertions compared to baseline GCNs?

주요 결과

Methods	Cora	Citeseer	Wine	Cancer	Digits
RBF SVM	59.7 (0.0)	60.2 (0.0)	94.1 (2.9)	91.7 (3.1)	86.9 (3.2)
SemiEmb	63.1 (0.1)	68.1 (0.1)	91.9 (0.1)	89.7 (0.1)	90.9 (0.1)
LDS	84.1 (0.4)	75.0 (0.4)	97.3 (0.4)	94.4 (1.9)	92.5 (0.7)
GCN	81.0 (0.2)	70.9 (0.3)	—	—	—
GAT	82.5 (0.4)	70.9 (0.4)	—	—	—
kNN-GCN	—	—	95.9 (0.9)	94.7 (1.2)	89.5 (1.3)
LDS*	83.9 (0.6)	74.8 (0.3)	96.9 (1.4)	93.4 (2.4)	90.8 (2.5)
DIAL-GNN	84.5 (0.3)	74.1 (0.2)	97.8 (0.6)	95.1 (1.0)	93.1 (0.5)

• DIAL-GNN consistently matches or outperforms state-of-the-art baselines across multiple datasets in both transductive and inductive settings.
• When graphs are not provided, DIAL-GNN outperforms kNN-GCN, demonstrating the value of joint graph learning with GNN parameters.
• Ablation shows significant performance drops when the iterative learning component is removed, emphasizing the importance of iterative graph refinement and joint optimization.
• The method remains robust to noisy graphs, with better performance than GCN under edge additions or deletions.