[Paper Review] Performance comparison of State-of-the-art Missing Value Imputation Algorithms on Some Bench mark Datasets

M. Naresh Kumar|arXiv (Cornell University)|Jul 22, 2013

Machine Learning and Data Classification22 references3 citations

TL;DR

This paper proposes the RNI-II imputation algorithm, which uses a novel indexing measure and distance computation to improve missing value estimation in datasets. Evaluated on UCI and KEEL benchmark datasets, RNI-II significantly outperforms KNNI, WKNNI, KMI, FKMI, and SVMI in enhancing classification accuracy of C4.5 and GA-C4.5 decision tree classifiers, with statistical significance (p < 0.05).

ABSTRACT

Decision making from data involves identifying a set of attributes that contribute to effective decision making through computational intelligence. The presence of missing values greatly influences the selection of right set of attributes and this renders degradation in classification accuracies of the classifiers. As missing values are quite common in data collection phase during field experiments or clinical trails appropriate handling would improve the classifier performance. In this paper we present a review of recently developed missing value imputation algorithms and compare their performance on some bench mark datasets.

Motivation & Objective

• To evaluate the performance of modern missing value imputation algorithms on real-world benchmark datasets.
• To address the degradation in classifier accuracy caused by missing values in clinical and temporal databases.
• To propose and validate the RNI-II algorithm as a superior imputation method for improving classification performance.
• To compare RNI-II with KNNI, WKNNI, KMI, FKMI, and SVMI using standard UCI and KEEL datasets.
• To establish statistical significance of RNI-II's performance gains using the Wilcoxon signed-rank test.

Proposed method

• RNI-II computes a novel indexing measure to assess data record similarity, enabling more accurate distance estimation between incomplete records.
• The algorithm uses this indexing measure to calculate proximity between data points, replacing traditional Euclidean distance in imputation.
• Imputation is performed by identifying the most similar records based on the new distance metric and assigning values from neighbors.
• For numerical attributes, the mean of similar neighbors is used; for nominal attributes, the mode is selected.
• The method integrates with decision tree classifiers (C4.5 and GA-C4.5) to evaluate imputation impact on classification accuracy.
• Statistical significance of performance differences is validated using the Wilcoxon signed-rank test at α = 0.05.

Experimental results

Research questions

• RQ1 How does the RNI-II imputation algorithm compare in performance to KNNI, WKNNI, KMI, FKMI, and SVMI on benchmark datasets?
• RQ2 Does RNI-II significantly improve the classification accuracy of C4.5 and GA-C4.5 classifiers when handling missing values?
• RQ3 Is the performance gain of RNI-II statistically significant compared to other imputation methods?
• RQ4 To what extent does the novel indexing measure in RNI-II enhance imputation accuracy over traditional distance-based methods?
• RQ5 How do different imputation strategies affect classifier performance across diverse real-world datasets?

Key findings

• RNI-II achieved a mean test accuracy of 85.65% on the AUS dataset using the GA-C4.5 classifier, significantly outperforming FKMI (81.45%) and KMI (81.16%).
• On the IRM dataset, RNI-II improved C4.5 accuracy by approximately 8% compared to FKMI, KNNI, and WKNNI.
• The Wilcoxon signed-rank test showed p-values of 0.02 or less when comparing RNI-II to KMI, FKMI, KNNI, and WKNNI, confirming statistical significance.
• For GA-C4.5, RNI-II achieved the highest positive rank sum (28) against all compared methods, with p-values ≤ 0.05.
• RNI-II did not achieve statistical significance when compared to SVMI (p = 0.37), though it still showed higher accuracy in most datasets.
• Visual analysis (Figures 2–6) confirmed that RNI-II consistently produced higher accuracy gains across all datasets, with bars above the baseline indicating consistent improvement.