[Paper Review] DyExplainer: Explainable Dynamic Graph Neural Networks

Tianchun Wang, Dongsheng Luo+3 more|arXiv (Cornell University)|Oct 25, 2023

Advanced Graph Neural Networks4 citations

TL;DR

DyExplainer introduces a novel, real-time explanation framework for dynamic Graph Neural Networks (GNNs) by integrating sparse structural and temporal attention mechanisms with contrastive learning and a buffer-based live-updating scheme. It enables faithful, consistent, and accurate model explanations across evolving graph snapshots, significantly improving link prediction performance while providing transparent, generalizable insights into dynamic GNN behavior.

ABSTRACT

Graph Neural Networks (GNNs) resurge as a trending research subject owing to their impressive ability to capture representations from graph-structured data. However, the black-box nature of GNNs presents a significant challenge in terms of comprehending and trusting these models, thereby limiting their practical applications in mission-critical scenarios. Although there has been substantial progress in the field of explaining GNNs in recent years, the majority of these studies are centered on static graphs, leaving the explanation of dynamic GNNs largely unexplored. Dynamic GNNs, with their ever-evolving graph structures, pose a unique challenge and require additional efforts to effectively capture temporal dependencies and structural relationships. To address this challenge, we present DyExplainer, a novel approach to explaining dynamic GNNs on the fly. DyExplainer trains a dynamic GNN backbone to extract representations of the graph at each snapshot, while simultaneously exploring structural relationships and temporal dependencies through a sparse attention technique. To preserve the desired properties of the explanation, such as structural consistency and temporal continuity, we augment our approach with contrastive learning techniques to provide priori-guided regularization. To model longer-term temporal dependencies, we develop a buffer-based live-updating scheme for training. The results of our extensive experiments on various datasets demonstrate the superiority of DyExplainer, not only providing faithful explainability of the model predictions but also significantly improving the model prediction accuracy, as evidenced in the link prediction task.

Motivation & Objective

• To address the lack of explainability in dynamic GNNs, which are increasingly used in real-world time-evolving applications.
• To provide model-level explanations that generalize across different dynamic GNN architectures, rather than being instance-specific.
• To ensure structural consistency and temporal continuity in explanations despite evolving graph structures and node features.
• To enhance model prediction accuracy through explainability-guided regularization using contrastive learning.
• To develop an efficient, on-the-fly explanation mechanism with minimal computational overhead for large-scale dynamic graphs.

Proposed method

• DyExplainer employs a dynamic GNN backbone to extract node representations at each time snapshot, capturing evolving graph structures and node attributes.
• It introduces a dual-attention mechanism: structural attention to identify key node relationships within each snapshot, and temporal attention to model long-term dependencies across consecutive snapshots.
• A sparse attention mechanism is used to focus on salient structural patterns, reducing noise and improving explanation fidelity.
• Contrastive learning is applied to regularize explanations by preserving structural consistency and temporal continuity, using positive and negative graph views derived from node representations.
• A buffer-based live-updating scheme maintains historical snapshots to model long-term temporal dynamics, enabling effective learning of evolving graph patterns.
• The explainer module is inserted after the GNN backbone and operates with minimal computational overhead, making it suitable for large-scale dynamic graphs.

Experimental results

Research questions

• RQ1 How can we provide faithful, real-time explanations for dynamic GNNs that account for both structural relationships and temporal evolution?
• RQ2 What techniques can ensure structural consistency and temporal continuity in explanations across evolving graph snapshots?
• RQ3 Can contrastive learning be effectively used to regularize explanations and improve both explainability and model performance in dynamic GNNs?
• RQ4 How does incorporating long-term temporal dependencies through a buffer-based scheme affect the accuracy and faithfulness of explanations?
• RQ5 To what extent can a generalizable, model-agnostic explainer be applied across diverse dynamic GNN architectures?

Key findings

• DyExplainer significantly improves link prediction accuracy on dynamic graph benchmarks, demonstrating that explainability-aware training enhances model performance.
• The method achieves faithful and consistent explanations by preserving structural and temporal properties through contrastive learning regularization.
• The sparse attention mechanism effectively identifies key subgraph structures and node relationships crucial for predictions at each snapshot.
• The buffer-based live-updating scheme enables the model to capture long-term temporal dependencies, outperforming methods relying only on immediate past snapshots.
• The integration of the explainer module incurs minimal computational overhead, making it scalable for large-scale dynamic graphs.
• DyExplainer provides generalizable, model-level explanations that are not limited to individual predictions, enabling broader interpretability across dynamic GNNs.