[Paper Review] Twin Learning for Similarity and Clustering: A Unified Kernel Approach

Zhao Kang, Chong Peng+1 more|arXiv (Cornell University)|May 1, 2017

Face and Expression Recognition20 citations

TL;DR

This paper proposes a unified kernel-based framework that jointly learns similarity matrices, cluster indicators, and optimal kernel combinations through twin learning, significantly improving clustering performance on real-world datasets by iteratively boosting each component using the others, with state-of-the-art results across multiple benchmarks including face and digit recognition datasets.

ABSTRACT

Many similarity-based clustering methods work in two separate steps including similarity matrix computation and subsequent spectral clustering. However, similarity measurement is challenging because it is usually impacted by many factors, e.g., the choice of similarity metric, neighborhood size, scale of data, noise and outliers. Thus the learned similarity matrix is often not suitable, let alone optimal, for the subsequent clustering. In addition, nonlinear similarity often exists in many real world data which, however, has not been effectively considered by most existing methods. To tackle these two challenges, we propose a model to simultaneously learn cluster indicator matrix and similarity information in kernel spaces in a principled way. We show theoretical relationships to kernel k-means, k-means, and spectral clustering methods. Then, to address the practical issue of how to select the most suitable kernel for a particular clustering task, we further extend our model with a multiple kernel learning ability. With this joint model, we can automatically accomplish three subtasks of finding the best cluster indicator matrix, the most accurate similarity relations and the optimal combination of multiple kernels. By leveraging the interactions between these three subtasks in a joint framework, each subtask can be iteratively boosted by using the results of the others towards an overall optimal solution. Extensive experiments are performed to demonstrate the effectiveness of our method.

Motivation & Objective

• To address the limitations of traditional two-step similarity-based clustering methods that separately compute similarity matrices and perform spectral clustering, which are sensitive to parameter choices and suboptimal similarity metrics.
• To overcome the challenge of nonlinear data structure by developing a method that operates directly in kernel spaces to capture complex, nonlinear relationships.
• To eliminate the need for manual kernel selection by integrating multiple kernel learning into the clustering framework, enabling automatic selection of optimal kernel combinations.
• To unify similarity learning, cluster indicator learning, and kernel weight learning into a single joint optimization framework where each component improves the others iteratively.

Proposed method

• The method formulates a joint optimization problem that simultaneously learns a similarity matrix Z and a cluster indicator matrix H, with a rank constraint on the Laplacian matrix of Z to enforce clustering structure.
• It leverages the self-expressive property, where each data point is represented as a linear combination of others, to learn global similarity relationships directly from data.
• The framework is extended to kernel spaces by applying the optimization in a reproducing kernel Hilbert space, enabling the capture of nonlinear data structures.
• A multiple kernel learning extension is introduced, where the final consensus kernel is learned as a weighted linear combination of multiple base kernels, with weights optimized jointly with similarity and clustering components.
• An efficient alternating optimization algorithm is designed to solve the non-convex problem, iteratively updating Z, H, and kernel weights until convergence.
• Theoretical connections are established with kernel k-means, k-means, and spectral clustering, showing that the proposed method generalizes and improves upon these classical approaches.

Experimental results

Research questions

• RQ1 Can a unified framework jointly optimize similarity learning, cluster indicator learning, and kernel selection to improve clustering performance?
• RQ2 How does the interaction between similarity learning and cluster indicator learning enhance the overall clustering quality compared to separate optimization?
• RQ3 To what extent can the proposed method outperform existing single-kernel and multiple-kernel clustering methods on real-world datasets with complex, nonlinear structures?
• RQ4 How robust is the method to parameter choices, particularly the regularization parameters α and β, across diverse datasets?

Key findings

• The proposed SCMK method achieves state-of-the-art performance on all benchmark datasets, with substantial improvements in accuracy, NMI, and purity over single-kernel and multiple-kernel baselines.
• On the JAFFE, AR, BA, TR11, and TR45 datasets, SCMK significantly outperforms other methods, demonstrating its effectiveness on challenging, real-world data with complex structures.
• The gap between the best and average results across single kernels highlights the critical impact of kernel choice, validating the need for automated kernel learning.
• The method shows strong robustness to parameter α across a wide range of values, though performance is more sensitive to β, suggesting careful tuning of β may be beneficial.
• Extensive experiments confirm that the joint optimization of similarity, clustering, and kernel weights leads to consistent performance gains, with each component improving through feedback from the others.