[Paper Review] Variational Graph Generator for Multi-View Graph Clustering

Jianpeng Chen, Yawen Ling+7 more|arXiv (Cornell University)|Oct 13, 2022

Complex Network Analysis Techniques4 citations

TL;DR

This paper proposes VGMGC, a variational graph generator for multi-view graph clustering that jointly learns view-specific and consensus graph structures using variational inference. By inferring a sparse, probabilistic consensus graph from multiple views and fusing it with view-specific features via a parameter-free GNN encoder, VGMGC achieves state-of-the-art performance on eight benchmark datasets, outperforming prior methods in clustering accuracy and normalized mutual information.

ABSTRACT

Multi-view graph clustering (MGC) methods are increasingly being studied due to the explosion of multi-view data with graph structural information. The critical point of MGC is to better utilize view-specific and view-common information in features and graphs of multiple views. However, existing works have an inherent limitation that they are unable to concurrently utilize the consensus graph information across multiple graphs and the view-specific feature information. To address this issue, we propose Variational Graph Generator for Multi-View Graph Clustering (VGMGC). Specifically, a novel variational graph generator is proposed to extract common information among multiple graphs. This generator infers a reliable variational consensus graph based on a priori assumption over multiple graphs. Then a simple yet effective graph encoder in conjunction with the multi-view clustering objective is presented to learn the desired graph embeddings for clustering, which embeds the inferred view-common graph and view-specific graphs together with features. Finally, theoretical results illustrate the rationality of the VGMGC by analyzing the uncertainty of the inferred consensus graph with the information bottleneck principle.Extensive experiments demonstrate the superior performance of our VGMGC over SOTAs. The source code is publicly available at https://github.com/cjpcool/VGMGC.

Motivation & Objective

• To address the limitation of existing multi-view graph clustering methods that fail to simultaneously utilize view-common graph structures and view-specific feature information.
• To develop a method that generates a robust, sparse consensus graph from multiple views using variational inference, incorporating both feature and graph-level commonality.
• To design a simple, parameter-free graph encoder that effectively fuses view-specific and consensus graph information with features for improved clustering.
• To theoretically justify the model's robustness using the information bottleneck principle, analyzing uncertainty in the inferred consensus graph.
• To empirically validate VGMGC's superiority over state-of-the-art methods across diverse multi-view graph clustering benchmarks.

Proposed method

• A priori assumption is formulated over multiple graphs to guide the variational inference process for consensus graph generation.
• A variational graph generator infers a sparse, discrete consensus graph by modeling the probability of each edge being part of the common structure, based on global features and multi-view graph inputs.
• A parameter-free message-passing mechanism is used in the graph encoder to aggregate information from both view-specific and inferred consensus graphs, preserving GNN expressiveness without redundant parameters.
• Multi-view representations are fused via a learnable aggregation mechanism that combines view-specific embeddings into a unified latent representation for clustering.
• The objective function combines clustering loss, reconstruction loss for the consensus graph, and a regularized term to encourage sparsity and uncertainty control via variational inference.
• Theoretical analysis using the information bottleneck principle demonstrates that the model minimizes uncertainty in the consensus graph, justifying its robustness.

Figure 1: The illustration of VGMGC. $\oplus$ denotes concatenating operation. We first infer a cross-view consensus graph $\mathbf{\hat{S}}$ via the variational graph generator, and then, the variational consensus graph is normalized, and we feed it with each attributed view-specific graph into the

Experimental results

Research questions

• RQ1 Can a variational inference-based graph generator effectively extract common topological structure across multiple views while preserving view-specific information?
• RQ2 How can a simple, parameter-free GNN encoder be designed to jointly embed view-specific and consensus graph structures with features?
• RQ3 Does the proposed method achieve better clustering performance than state-of-the-art multi-view graph clustering approaches on diverse datasets?
• RQ4 How sensitive is the model to hyperparameters such as the temperature τ and the consensus weight ρ?
• RQ5 To what extent does the inferred consensus graph reduce uncertainty and improve robustness, as justified by information bottleneck theory?

Key findings

• VGMGC achieves state-of-the-art performance on eight multi-view graph clustering datasets, outperforming existing methods in both clustering accuracy and normalized mutual information.
• On the DBLP dataset, VGMGC achieves a 2.5% improvement in NMI over the best single-view result, demonstrating the effectiveness of multi-view fusion.
• The model shows stable performance across different values of hyperparameters ρ and τ, with NMI varying by less than 3% and ACC within 3% across tested ranges.
• Convergence analysis confirms that all loss components stabilize after approximately 100 training epochs, indicating reliable training dynamics.
• Visualization via t-SNE shows that the fused representation (Z̄) successfully separates ambiguous clusters present in individual views, especially outperforming the third view on DBLP.
• Oversmoothing analysis reveals that performance improves with higher message-passing order, indicating that VGMGC effectively mitigates feature over-smoothing in deep GNNs.

Figure 2: The Bayesian net of our variational graph generator about variables: $\mathbf{A}^{v}$ , $\mathbf{\hat{S}}$ , $\mathbf{\overline{X}}$ and $\mathbf{X}^{v}$ .