[Paper Review] Decoupled Contrastive Multi-View Clustering with High-Order Random Walks

Yiding Lu, Yijie Lin+4 more|arXiv (Cornell University)|Aug 22, 2023

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes DIVIDE, a novel contrastive multi-view clustering method that uses high-order random walks on affinity graphs to identify false negatives and false positives, enabling global, robust pair selection. By decoupling intra- and inter-view contrastive learning in separate embedding spaces, DIVIDE improves clustering performance and robustness to missing views, outperforming nine SOTA methods on four benchmarks in both complete and incomplete settings.

ABSTRACT

In recent, some robust contrastive multi-view clustering (MvC) methods have been proposed, which construct data pairs from neighborhoods to alleviate the false negative issue, i.e., some intra-cluster samples are wrongly treated as negative pairs. Although promising performance has been achieved by these methods, the false negative issue is still far from addressed and the false positive issue emerges because all in- and out-of-neighborhood samples are simply treated as positive and negative, respectively. To address the issues, we propose a novel robust method, dubbed decoupled contrastive multi-view clustering with high-order random walks (DIVIDE). In brief, DIVIDE leverages random walks to progressively identify data pairs in a global instead of local manner. As a result, DIVIDE could identify in-neighborhood negatives and out-of-neighborhood positives. Moreover, DIVIDE embraces a novel MvC architecture to perform inter- and intra-view contrastive learning in different embedding spaces, thus boosting clustering performance and embracing the robustness against missing views. To verify the efficacy of DIVIDE, we carry out extensive experiments on four benchmark datasets comparing with nine state-of-the-art MvC methods in both complete and incomplete MvC settings.

Motivation & Objective

• To address the persistent false negative (FN) and false positive (FP) issues in contrastive multi-view clustering caused by local neighborhood-based pair selection.
• To overcome the under- and over-rectification problems in existing neighborhood-based methods that treat all in-neighborhood samples as positive and all out-of-neighborhood as negative.
• To improve clustering performance by preserving view-specific information while enhancing cross-view consistency through a decoupled learning architecture.
• To enable robust performance under incomplete multi-view settings where some views may be missing during inference.

Proposed method

• DIVIDE employs multi-step random walks on an affinity graph to compute high-order affinities between anchors and their neighbors, enabling global identification of data pairs beyond first-order neighborhoods.
• High-order affinities allow the model to detect in-neighborhood inter-cluster samples as negatives (reducing FPs) and out-of-neighborhood intra-cluster samples as positives (reducing FNs).
• A decoupled contrastive learning framework performs intra-view and inter-view contrastive learning in separate embedding spaces using a view-specific Siamese encoder and a cross-view decoder.
• The framework uses an online and target encoder with EMA-based momentum updates to stabilize training, with BatchNorm and ReLU layers in fully connected networks.
• The random walk process is applied starting from epoch 100, with transition matrices visualized to demonstrate progressive FN detection over time.
• The method integrates a cross-view decoder to reconstruct features from one view using representations from another, promoting cross-view consistency without mixing intra-view and inter-view learning.

Experimental results

Research questions

• RQ1 Can high-order random walks on affinity graphs effectively reduce false negative and false positive issues in contrastive multi-view clustering?
• RQ2 How does global pair selection via random walks compare to local neighborhood-based pair selection in terms of clustering accuracy and robustness?
• RQ3 To what extent does decoupling intra- and inter-view contrastive learning improve clustering performance and view-specific representation learning?
• RQ4 How does the proposed method perform under incomplete multi-view settings where some views are missing during inference?

Key findings

• DIVIDE achieves state-of-the-art performance on four benchmark datasets—Scene-15, Caltech101, Reuters, and LandUse21—under both complete and incomplete multi-view settings.
• On the Reuters dataset, the model reaches 59.1% accuracy at epoch 200 with random walk-based false negative correction, compared to 56.2% without it, demonstrating the effectiveness of the proposed FN identification.
• Visualization of t-SNE features shows that clusters become significantly more compact and well-separated after applying the random walk-based correction, especially after 100 training epochs.
• The transition matrices from multi-step random walks progressively align with ground-truth class labels, confirming the method’s ability to detect intra-cluster samples beyond first-order neighborhoods.
• The decoupled architecture preserves view-specific information while enhancing cross-view consistency, leading to improved clustering performance compared to joint learning frameworks.
• Extensive ablation studies confirm that both the random walk mechanism and the decoupled contrastive learning design are essential for the method’s superior performance.