[论文解读] Distance and Similarity Measures Effect on the Performance of K-Nearest Neighbor Classifier -- A Review

V. B. Surya Prasath, Haneen Arafat Abu Alfeilat+5 more|arXiv (Cornell University)|Aug 14, 2017

Machine Learning and Data Classification参考文献 72被引用 97

一句话总结

这项综述评估了大量距离/相似性度量（最多54种）在 KNN 上对众多真实世界数据集（包括噪声）的表现，以识别哪些度量可以提供最佳准确性和鲁棒性。

ABSTRACT

The K-nearest neighbor (KNN) classifier is one of the simplest and most common classifiers, yet its performance competes with the most complex classifiers in the literature. The core of this classifier depends mainly on measuring the distance or similarity between the tested examples and the training examples. This raises a major question about which distance measures to be used for the KNN classifier among a large number of distance and similarity measures available? This review attempts to answer this question through evaluating the performance (measured by accuracy, precision and recall) of the KNN using a large number of distance measures, tested on a number of real-world datasets, with and without adding different levels of noise. The experimental results show that the performance of KNN classifier depends significantly on the distance used, and the results showed large gaps between the performances of different distances. We found that a recently proposed non-convex distance performed the best when applied on most datasets comparing to the other tested distances. In addition, the performance of the KNN with this top performing distance degraded only about $20\%$ while the noise level reaches $90\%$, this is true for most of the distances used as well. This means that the KNN classifier using any of the top $10$ distances tolerate noise to a certain degree. Moreover, the results show that some distances are less affected by the added noise comparing to other distances.

研究动机与目标

• 评估在真实世界数据集上，距离/相似性度量的选择如何影响 KNN 分类准确率、精确度和召回率。
• 比较涵盖八大类共54种度量的大量距离度量，以识别表现最佳者。
• 通过观察在噪声水平增加时的性能下降来评估 KNN 的鲁棒性。
• 确定对噪声不敏感、因此在 KNN 分类中更鲁棒的距离度量。

提出的方法

• 汇编并将54种距离/相似度度量归类为八大类。
• 在大量真实世界数据集上评估 KNN 的表现（准确率、精确度、召回率），并测定不同噪声水平。
• 通过加入不同的噪声水平，最高至90%，并观察性能变化来分析鲁棒性。
• 确定表现最佳的距离，并比较它们的噪声容忍度。
• 总结结果在不同数据集之间的对比，并讨论对实际应用的影响。

实验结果

研究问题

• RQ1 在多样化数据集上，最大化 KNN 准确性（及相关指标）的最佳距离度量是什么？
• RQ2 当数据中存在噪声时，哪种距离度量能维持最佳性能？
• RQ3 哪些距离度量在 KNN 分类中对噪声最具鲁棒性？

主要发现

• KNN 的性能显著依赖所使用的距离度量，不同度量之间存在较大差距。
• 最近提出的非凸距离在大多数数据集上通常能得到最佳结果。
• 当噪声达到90%时，表现最好的距离大约下降20%，表明具有相当的鲁棒性。
• 许多前十名距离在一定程度上容忍噪声。
• 某些距离对附加噪声有更强的抗性，凸显不同度量之间的鲁棒性差异。
• Hassanat distance 在多次比较中被凸显为表现出色。