[Paper Review] Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks

Yanbang Wang, Yen‐Yu Chang+3 more|arXiv (Cornell University)|Jan 15, 2021

Advanced Graph Neural NetworksComputer Science70 references55 citations

TL;DR

CAW-N learns inductive representations for temporal networks by using Causal Anonymous Walks to capture temporal motifs, anonymize node identities with relative positions, and encode walks with neural networks for accurate inductive link prediction.

ABSTRACT

Temporal networks serve as abstractions of many real-world dynamic systems. These networks typically evolve according to certain laws, such as the law of triadic closure, which is universal in social networks. Inductive representation learning of temporal networks should be able to capture such laws and further be applied to systems that follow the same laws but have not been unseen during the training stage. Previous works in this area depend on either network node identities or rich edge attributes and typically fail to extract these laws. Here, we propose Causal Anonymous Walks (CAWs) to inductively represent a temporal network. CAWs are extracted by temporal random walks and work as automatic retrieval of temporal network motifs to represent network dynamics while avoiding the time-consuming selection and counting of those motifs. CAWs adopt a novel anonymization strategy that replaces node identities with the hitting counts of the nodes based on a set of sampled walks to keep the method inductive, and simultaneously establish the correlation between motifs. We further propose a neural-network model CAW-N to encode CAWs, and pair it with a CAW sampling strategy with constant memory and time cost to support online training and inference. CAW-N is evaluated to predict links over 6 real temporal networks and uniformly outperforms previous SOTA methods by averaged 10% AUC gain in the inductive setting. CAW-N also outperforms previous methods in 4 out of the 6 networks in the transductive setting.

Motivation & Objective

• Motivate learning representations for temporal networks that generalize inductively to unseen nodes and motifs.
• Capture temporal network laws (e.g., triadic closure, feedforward loops) via causal motifs without depending on node identities.
• Develop CAWs with causality extraction and set-based anonymization to enable inductive learning.
• Propose CAW-N to encode CAWs and predict temporal links with constant-memory sampling for online use.
• Demonstrate state-of-the-art performance on six real temporal networks in inductive and transductive settings.

Proposed method

• Introduce Causal Anonymous Walks (CAWs) that backtrack along temporal links to encode causal temporal motifs.
• Apply set-based anonymization by counting node appearances at each position across multiple walks to create I_CAW(w;{S_u,S_v}), preserving motif correlations while removing identities.
• Encode CAWs with CAW-N: encode each anonymized walk via an RNN using f1(I_CAW) and time encoding f2(t_{i-1}-t_i); aggregate via mean-pooling or self-attention.
• Use time-feature encoding via random Fourier features to represent time differences.
• Train CAW-N with binary cross-entropy loss for link prediction, enabling online learning with a CAW sampling strategy of constant memory (O(tau/alpha)).
• Extensions include incorporating attributes and directed links by augmenting encodings with X_i, and using efficient CAW sampling with alpha-based temporal decay.

Experimental results

Research questions

• RQ1 Can inductive representations of temporal networks be learned without relying on node identities by capturing temporal motif correlations?
• RQ2 Do CAWs and the CAW-N encoder generalize to unseen nodes and new links while preserving temporal dynamics?
• RQ3 How does CAW-N perform on inductive and transductive link prediction across multiple real temporal networks compared to state-of-the-art baselines?
• RQ4 What is the impact of CAW sampling parameters (M, m, alpha) on predictive performance and online scalability?
• RQ5 How do attributes and directed edges influence the CAW-N model’s effectiveness?

Key findings

• CAW-N-mean and CAW-N-attn achieve strong inductive link-prediction performance, outperforming baselines across six real temporal networks in the inductive setting.
• In inductive tasks, especially for new-vs-new links, CAW-N substantially surpasses all baselines on all datasets.
• In transductive settings, CAW-N still outperforms most baselines on four of six datasets and matches or nears best on others.
• The anonymization I_CAW(w;{S_u,S_v}) effectively preserves motif correlations while removing node identities, enabling robust inductive generalization.
• A small number of CAWs with an appropriate alpha value suffices for competitive performance, and the approach supports online learning with constant memory.