[Paper Review] Comparing K-Nearest Neighbors and Potential Energy Method in classification problem. A case study using KNN applet by E.M. Mirkes and real life benchmark data sets

Yanshan Shi|arXiv (Cornell University)|Nov 5, 2012

Face and Expression Recognition5 references3 citations

TL;DR

This study compares K-Nearest Neighbors (KNN) and Potential Energy (PE) methods—using Yukawa and Gaussian potentials—for supervised classification. Using E.M. Mirkes' KNN applet and real benchmark datasets, it finds that KNN and PE with Gaussian potential perform similarly, while PE with Yukawa potential underperforms KNN in high-density data distributions.

ABSTRACT

K-nearest neighbors (KNN) method is used in many supervised learning classification problems. Potential Energy (PE) method is also developed for classification problems based on its physical metaphor. The energy potential used in the experiments are Yukawa potential and Gaussian Potential. In this paper, I use both applet and MATLAB program with real life benchmark data to analyze the performances of KNN and PE method in classification problems. The results show that in general, KNN and PE methods have similar performance. In particular, PE with Yukawa potential has worse performance than KNN when the density of the data is higher in the distribution of the database. When the Gaussian potential is applied, the results from PE and KNN have similar behavior. The indicators used are correlation coefficients and information gain.

Motivation & Objective

• To evaluate and compare the classification performance of K-Nearest Neighbors (KNN) and Potential Energy (PE) methods in real-world datasets.
• To investigate how different potential functions—Yukawa and Gaussian—affect the classification accuracy of the PE method.
• To analyze the impact of data density on the relative performance of KNN and PE methods.
• To validate findings using both interactive KNN applet and MATLAB-based implementation on benchmark datasets.
• To use correlation coefficients and information gain as quantitative indicators for performance comparison.

Proposed method

• Employs E.M. Mirkes' KNN applet for interactive visualization and classification of benchmark datasets.
• Implements the Potential Energy method using Yukawa and Gaussian potentials as energy functions for classification.
• Applies both KNN and PE methods to the same set of real-life benchmark datasets to ensure fair comparison.
• Uses MATLAB for reproducible classification experiments and performance evaluation.
• Employs correlation coefficients and information gain as quantitative metrics to assess model performance.
• Analyzes classification results across varying data density conditions to assess robustness of each method.

Experimental results

Research questions

• RQ1 How do KNN and Potential Energy methods compare in terms of classification accuracy across real benchmark datasets?
• RQ2 Does the choice of potential function (Yukawa vs. Gaussian) significantly affect the performance of the PE method?
• RQ3 How does data density influence the relative performance of KNN and PE methods?
• RQ4 Are the results from the KNN applet consistent with those from the MATLAB-based implementation?
• RQ5 Do correlation coefficients and information gain yield aligned conclusions about model performance?

Key findings

• KNN and PE with Gaussian potential exhibit very similar classification performance across the tested benchmark datasets.
• PE with Yukawa potential consistently underperforms KNN, particularly in datasets with high data density.
• The correlation coefficients between predicted and actual class labels are comparable for KNN and Gaussian PE, indicating strong consistency.
• Information gain values are similar for KNN and Gaussian PE, suggesting equivalent feature relevance detection.
• The performance gap between KNN and Yukawa PE widens as data density increases, indicating sensitivity to clustering.
• The KNN applet provides reliable results that align with MATLAB-based implementations, validating the experimental approach.