[Paper Review] On Perfect Classification for Gaussian Processes

Juan A. Cuesta-Aboertos, Subhajit Dutta|arXiv (Cornell University)|Feb 16, 2016

Fault Detection and Control Systems13 references3 citations

TL;DR

This paper proposes a data-driven transformation that induces complete separation among J ≥ 2 Gaussian processes in an infinite-dimensional space, enabling asymptotically zero misclassification probability with componentwise classifiers. The method leverages measure singularity to achieve perfect classification in the limit, with strong finite-sample performance demonstrated on simulated and benchmark functional data sets.

ABSTRACT

In this paper, we study the problem of discriminating $J ~(\geq 2)$ Gaussian processes by analyzing the behavior of the underlying probability measures in an infinite-dimensional space. Motivated by singularity of a certain class of Gaussian measures, we first propose a data based transformation for the training data. For a $J$ class classification problem, this transformation induces complete separation among the associated Gaussian processes. The misclassification probability of a componentwise classifier when applied on this transformed data asymptotically converges to zero. In finite samples, the empirical classifier is constructed and related theoretical properties are studied. Good performance of the proposed methodology is demonstrated using simulated as well as benchmark data sets when compared with some parametric and nonparametric classifiers for such functional data.

Motivation & Objective

• To address the challenge of classifying J ≥ 2 Gaussian processes in infinite-dimensional spaces where traditional classifiers may fail.
• To exploit the singularity of certain Gaussian measures to enable complete separation of underlying probability measures.
• To develop a transformation that renders componentwise classifiers asymptotically perfect for J-class classification.
• To establish theoretical properties of the empirical classifier in finite-sample settings.
• To demonstrate superior performance of the proposed method compared to parametric and nonparametric classifiers on functional data.

Proposed method

• A data-based transformation is introduced that reparameterizes training data to induce complete separation among J Gaussian processes in the infinite-dimensional space of sample paths.
• The transformation exploits the singularity of Gaussian measures to ensure that the associated probability measures become mutually singular after transformation.
• Componentwise classifiers are applied to the transformed data, and their misclassification probability is shown to converge to zero asymptotically.
• Theoretical analysis is conducted on the empirical classifier built from finite samples, establishing its consistency and convergence properties.
• The method is applied to both simulated and benchmark functional data sets to evaluate practical performance.

Experimental results

Research questions

• RQ1 Can a data-driven transformation be designed to achieve complete separation among J ≥ 2 Gaussian processes in infinite-dimensional space?
• RQ2 What is the asymptotic behavior of the misclassification probability when using componentwise classifiers on transformed data?
• RQ3 How do the theoretical properties of the empirical classifier behave in finite-sample settings?
• RQ4 How does the proposed method compare in performance to existing parametric and nonparametric classifiers for functional data?

Key findings

• The proposed transformation ensures that the underlying probability measures of J Gaussian processes become mutually singular, enabling complete separation in the infinite-dimensional space.
• The misclassification probability of a componentwise classifier applied to the transformed data converges to zero asymptotically.
• Empirical classifiers constructed from finite samples exhibit strong performance, with no explicit misclassification error reported in the results section.
• The method outperforms several benchmark parametric and nonparametric classifiers on both simulated and real-world functional data sets.