[Paper Review] Deep Temporal Graph Clustering

Meng Liu, Yue Liu+5 more|arXiv (Cornell University)|May 18, 2023

Complex Network Analysis Techniques25 citations

TL;DR

This paper proposes TGC, a general framework for unsupervised clustering on temporal graphs that adapts deep clustering modules to interaction-sequence batching, achieving better scalability and clustering performance than static-graph methods.

ABSTRACT

Deep graph clustering has recently received significant attention due to its ability to enhance the representation learning capabilities of models in unsupervised scenarios. Nevertheless, deep clustering for temporal graphs, which could capture crucial dynamic interaction information, has not been fully explored. It means that in many clustering-oriented real-world scenarios, temporal graphs can only be processed as static graphs. This not only causes the loss of dynamic information but also triggers huge computational consumption. To solve the problem, we propose a general framework for deep Temporal Graph Clustering called TGC, which introduces deep clustering techniques to suit the interaction sequence-based batch-processing pattern of temporal graphs. In addition, we discuss differences between temporal graph clustering and static graph clustering from several levels. To verify the superiority of the proposed framework TGC, we conduct extensive experiments. The experimental results show that temporal graph clustering enables more flexibility in finding a balance between time and space requirements, and our framework can effectively improve the performance of existing temporal graph learning methods. The code is released: https://github.com/MGitHubL/Deep-Temporal-Graph-Clustering.

Motivation & Objective

• Motivate the need for temporal-graph clustering that preserves dynamic interaction information.
• Propose a general, scalable framework (TGC) that adapts clustering techniques to batch-based temporal data.
• Enable effective node clustering on temporal graphs while mitigating memory bottlenecks.
• Demonstrate the framework’s transferability to existing temporal graph learning models.

Proposed method

• Adapts two classical deep clustering components to temporal graphs: node-level distribution and batch-level reconstruction.
• Uses a Hawkes-process-inspired temporal loss to model interaction dynamics.
• Implements node-level soft assignments q and a target distribution p to drive clustering with KL divergence loss L_node.
• Implements a batch-level pseudo-reconstruction loss L_batch based on cosine similarity to preserve local adjacency relations within batches.
• Overall clustering loss L_clu = L_node + L_batch, combined with temporal loss L_tem in an end-to-end objective L = sum over edges (L_tem + L_clu).
• Analyzes complexity as O(|E|) and demonstrates memory benefits over O(N^2) static adjacency-matrix clustering.

Experimental results

Research questions

• RQ1 How can clustering modules be reformulated to work with the batch-processing pattern of temporal graphs?
• RQ2 Do temporal graphs offer memory and scalability advantages for clustering compared to static graphs?
• RQ3 Can the proposed TGC framework improve clustering performance when applied to existing temporal-graph models?
• RQ4 What are the practical data, complexity, and transferability considerations for temporal graph clustering?
• RQ5 What are the limitations and data requirements hindering temporal graph clustering development?

Key findings

• TGC enables memory-efficient temporal clustering with complexity around O(|E|), often far below static O(N^2) methods.
• Temporal batching allows effective clustering on large-scale temporal graphs (e.g., arXivAI/arXivCS) where static methods struggle or run out of memory.
• TGC improves clustering performance across multiple temporal-graph baselines (e.g., HTNE, TGN, TREND) by incorporating temporal dynamics into clustering losses.
• Temporal graph clustering shows flexibility in balancing time vs. space requirements via batch size tuning.
• Two novel datasets (arXivAI, arXivCS) demonstrate applicability to large temporal graphs with node labels derived from domains, enabling clustering evaluation.
• Experiments indicate TGC often yields leading aggregate results and mitigates memory bottlenecks on large graphs.