[Paper Review] Min-Max Kernels.

Ping Li|arXiv (Cornell University)|Mar 5, 2015

Face and Expression Recognition23 references3 citations

TL;DR

This paper proposes a linearization of the min-max kernel via a novel 0-bit hashing scheme, enabling efficient large-scale classification using linear models like SVM and logistic regression. The method discards the theoretically unbounded $t^*$ component from consistent weighted sampling while preserving essential similarity information, achieving competitive performance on diverse nonnegative data sets.

ABSTRACT

The min-max kernel is a generalization of the popular resemblance kernel (which is designed for binary data). In this paper, we demonstrate, through an extensive classification study using kernel machines, that the min-max kernel often provides an effective measure of similarity for nonnegative data. As the min-max kernel is nonlinear and might be difficult to be used for industrial applications with massive data, we show that the min-max kernel can be linearized via hashing techniques. This allows practitioners to apply min-max kernel to large-scale applications using well matured linear algorithms such as linear SVM or logistic regression. The previous remarkable work on consistent weighted sampling (CWS) produces samples in the form of ($i^*, t^*$) where the $i^*$ records the location (and in fact also the weights) information analogous to the samples produced by classical minwise hashing on binary data. Because the $t^*$ is theoretically unbounded, it was not immediately clear how to effectively implement CWS for building large-scale linear classifiers. In this paper, we provide a simple solution by discarding $t^*$ (which we refer to as the scheme). Via an extensive empirical study, we show that this 0-bit scheme does not lose essential information. We then apply the CWS for building linear classifiers to approximate min-max kernel classifiers, as extensively validated on a wide range of publicly available classification datasets. We expect this work will generate interests among data mining practitioners who would like to efficiently utilize the nonlinear information of non-binary and nonnegative data.

Motivation & Objective

• To address the challenge of applying the nonlinear min-max kernel to large-scale industrial applications with massive nonnegative data.
• To overcome the impracticality of handling the unbounded $t^*$ component in consistent weighted sampling (CWS) for large-scale linear classifiers.
• To develop a practical hashing-based linearization of the min-max kernel that retains nonlinear similarity information without storing $t^*$.
• To validate the effectiveness of the proposed 0-bit scheme in approximating min-max kernel classifiers using standard linear algorithms.
• To enable data mining practitioners to efficiently leverage nonlinear similarity for non-binary, nonnegative data in scalable settings.

Proposed method

• Adapting consistent weighted sampling (CWS) to generate samples in the form ($i^*$, $t^*$) for nonnegative data, analogous to minwise hashing for binary data.
• Proposing a 0-bit scheme that discards the $t^*$ component, which is theoretically unbounded and impractical for large-scale deployment.
• Using only the index $i^*$ (location and weight information) to represent samples, effectively reducing storage and computational overhead.
• Applying the resulting hash vectors to train linear classifiers such as linear SVM and logistic regression as approximations of min-max kernel classifiers.
• Leveraging the nonlinear nature of the min-max kernel through a linearized representation that preserves essential similarity structure.
• Validating the method through extensive empirical evaluation on diverse public classification datasets using standard machine learning pipelines.

Experimental results

Research questions

• RQ1 Can the min-max kernel be effectively linearized for large-scale applications without significant loss of similarity information?
• RQ2 Is discarding the $t^*$ component in consistent weighted sampling viable for building accurate linear classifiers on nonnegative data?
• RQ3 How well does the 0-bit hashing scheme approximate the performance of full min-max kernel classifiers in practice?
• RQ4 Can linear models trained on the proposed hash representations achieve competitive accuracy compared to kernel methods on nonnegative data?
• RQ5 What is the scalability and efficiency gain of using the 0-bit scheme over direct min-max kernel computation in large-scale settings?

Key findings

• The 0-bit scheme, which discards the unbounded $t^*$, preserves essential similarity information and enables effective linear classification.
• Linear classifiers trained on the proposed hash representations achieve performance competitive with full min-max kernel classifiers on a wide range of nonnegative data sets.
• The method enables scalable deployment of min-max kernel-based similarity in industrial applications by leveraging mature linear algorithms.
• Empirical evaluation confirms that the linearized approach maintains high accuracy while drastically reducing storage and computational costs.
• The approach is robust across diverse public datasets, demonstrating generalizability beyond specific data types or domains.
• The results suggest that nonlinear similarity from the min-max kernel can be efficiently harnessed in large-scale settings through linearization without significant performance degradation.