[Paper Review] Attributed Graph Clustering: A Deep Attentional Embedding Approach

Chun Wang, Shirui Pan+4 more|arXiv (Cornell University)|Jun 15, 2019

Advanced Graph Neural Networks18 references48 citations

TL;DR

DAEGC jointly learns a goal-directed graph embedding and clustering for attributed graphs using a graph attention autoencoder and a self-training clustering module, achieving superior clustering performance over state-of-the-art baselines.

ABSTRACT

Graph clustering is a fundamental task which discovers communities or groups in networks. Recent studies have mostly focused on developing deep learning approaches to learn a compact graph embedding, upon which classic clustering methods like k-means or spectral clustering algorithms are applied. These two-step frameworks are difficult to manipulate and usually lead to suboptimal performance, mainly because the graph embedding is not goal-directed, i.e., designed for the specific clustering task. In this paper, we propose a goal-directed deep learning approach, Deep Attentional Embedded Graph Clustering (DAEGC for short). Our method focuses on attributed graphs to sufficiently explore the two sides of information in graphs. By employing an attention network to capture the importance of the neighboring nodes to a target node, our DAEGC algorithm encodes the topological structure and node content in a graph to a compact representation, on which an inner product decoder is trained to reconstruct the graph structure. Furthermore, soft labels from the graph embedding itself are generated to supervise a self-training graph clustering process, which iteratively refines the clustering results. The self-training process is jointly learned and optimized with the graph embedding in a unified framework, to mutually benefit both components. Experimental results compared with state-of-the-art algorithms demonstrate the superiority of our method.

Motivation & Objective

• Motivate graph clustering on attributed graphs by jointly exploiting structure and node content.
• Develop a goal-directed, end-to-end framework that integrates embedding learning with clustering.
• Leverage graph attention to capture high-order neighborhood information.
• Introduce a self-training clustering module to supervise embedding learning.
• Demonstrate superiority over state-of-the-art baselines on benchmark datasets.

Proposed method

• Propose a graph attentional autoencoder to fuse graph structure and node attributes into a latent representation.
• Use a two-layer graph attention encoder with attention coefficients computed from node attributes and topology, incorporating high-order neighbors via a proximity matrix M.
• Decode with an inner product to reconstruct the adjacency matrix and minimize reconstruction loss L_r.
• Introduce a self-training clustering module that minimizes KL divergence L_c between soft assignments Q and a target distribution P, derived from a Student-t similarity metric.
• Jointly optimize L = L_r + γ L_c to learn embeddings and clustering simultaneously.
• Initialize clustering with k-means on the learned embedding and update cluster centers μ during training.

Experimental results

Research questions

• RQ1 How can we design a goal-directed embedding framework that improves graph clustering for attributed graphs?
• RQ2 Can a graph attention-based encoder effectively fuse structure and content to produce clustering-friendly embeddings?
• RQ3 Does a self-training clustering module improve clustering quality by supervising embedding learning?
• RQ4 What performance gains can be achieved over state-of-the-art graph clustering methods on standard datasets?

Key findings

• DAEGC outperforms baselines on standard attributed graph clustering benchmarks across ACC, NMI, F-score, and ARI.
• Integrating structure and content via graph attention improves clustering performance beyond structure-only or content-only methods.
• The self-training clustering component provides supervision that guides embedding optimization toward better clustering.
• On Cora, DAEGC achieves ACC 0.704, NMI 0.528, F-score 0.682, ARI 0.496.
• On Citeseer, DAEGC achieves ACC 0.671, NMI 0.266, F-score 0.659, ARI 0.278.
• On PubMed, DAEGC achieves ACC 0.671, NMI 0.266, F-score 0.659, ARI 0.278.