[Paper Review] Deep Contrastive Graph Representation via Adaptive Homotopy Learning

Rui Zhang, Chengjun Lu+2 more|arXiv (Cornell University)|Jun 17, 2021

Advanced Graph Neural Networks51 references4 citations

TL;DR

This paper proposes Adaptive Homotopy Contrastive Learning (AHCL), a novel framework that enables deep contrastive representation learning in unsupervised settings by adaptively learning soft labels via Maclaurin duality, eliminating reliance on prior labels. The method constructs a deep Laplacian graph from adaptive affinity matrices, achieving state-of-the-art clustering performance on benchmark datasets without any supervision or manual hyperparameter tuning.

ABSTRACT

Homotopy model is an excellent tool exploited by diverse research works in the field of machine learning. However, its flexibility is limited due to lack of adaptiveness, i.e., manual fixing or tuning the appropriate homotopy coefficients. To address the problem above, we propose a novel adaptive homotopy framework (AH) in which the Maclaurin duality is employed, such that the homotopy parameters can be adaptively obtained. Accordingly, the proposed AH can be widely utilized to enhance the homotopy-based algorithm. In particular, in this paper, we apply AH to contrastive learning (AHCL) such that it can be effectively transferred from weak-supervised learning (given label priori) to unsupervised learning, where soft labels of contrastive learning are directly and adaptively learned. Accordingly, AHCL has the adaptive ability to extract deep features without any sort of prior information. Consequently, the affinity matrix formulated by the related adaptive labels can be constructed as the deep Laplacian graph that incorporates the topology of deep representations for the inputs. Eventually, extensive experiments on benchmark datasets validate the superiority of our method.

Motivation & Objective

• To overcome the lack of adaptiveness in traditional homotopy methods, which rely on fixed, manually tuned coefficients.
• To enable homotopy-based algorithms to function effectively in unsupervised representation learning without prior label information.
• To develop a framework that adaptively learns soft labels and affinity relationships from data, forming a deep Laplacian graph that captures the topology of learned representations.
• To improve deep feature extraction and representation learning in contrastive learning by integrating adaptive weight learning and autoencoder reconstruction.

Proposed method

• Introduces an adaptive homotopy framework (AH) that uses Maclaurin duality to automatically learn optimal homotopy parameters, replacing manual coefficient tuning.
• Applies the AH framework to contrastive learning, resulting in Adaptive Homotopy Contrastive Learning (AHCL), which learns soft labels end-to-end in an unsupervised manner.
• Constructs a deep Laplacian graph using the affinity matrix derived from adaptive soft labels, encoding the topological structure of deep representations.
• Employs a Siamese encoder with a residual block architecture to extract deep embeddings, followed by an unbalanced autoencoder to reconstruct input data.
• Uses Mean Squared Error (MSE) as the reconstruction loss to prevent feature collapse and stabilize training.
• Leverages the adaptive affinity matrix as a learnable, data-driven similarity measure to guide representation learning.

Experimental results

Research questions

• RQ1 Can homotopy-based methods be made adaptive to eliminate reliance on pre-specified coefficients in representation learning?
• RQ2 Can contrastive learning be effectively transferred from weakly supervised to fully unsupervised settings using adaptive label learning?
• RQ3 Can adaptive soft labels derived from deep features form a meaningful and topologically informative Laplacian graph?
• RQ4 Does the integration of adaptive homotopy with autoencoder reconstruction improve feature representation and prevent collapse?

Key findings

• AHCL achieves state-of-the-art performance on five benchmark datasets, outperforming nine baseline methods in both ACC and NMI across all datasets.
• On the UMIST dataset, AHCL achieves 90.78% ACC and 95.47% NMI, significantly outperforming the second-best method (SC-Y at 89.33% ACC).
• On the FASHION dataset, AHCL achieves 96.45% ACC and 99.32% NMI, demonstrating strong performance on complex, real-world data.
• The ablation study confirms that AHCL significantly improves clustering performance over KM-Z, SC-Z, and SC-Y baselines, with the highest gains observed in deep feature and affinity matrix learning.
• Convergence analysis shows rapid and stable convergence within 60 epochs across all datasets, indicating robust training dynamics due to adaptive weight learning.