[Paper Review] Autoencoder, Principal Component Analysis and Support Vector Regression for Data Imputation

Vukosi Marivate, Fulufhelo V. Nelwamodo+1 more|ArXiv.org|Sep 16, 2007

Neural Networks and Applications20 references17 citations

TL;DR

This paper proposes a hybrid data imputation framework combining autoencoders, principal component analysis (PCA), and support vector regression (SVR), optimized via a genetic algorithm. It achieves up to 97.4% accuracy in imputing missing values across variables, demonstrating that PCA enhances autoencoder performance while SVR shows strong potential for future imputation research.

ABSTRACT

Data collection often results in records that have missing values or variables. This investigation compares 3 different data imputation models and identifies their merits by using accuracy measures. Autoencoder Neural Networks, Principal components and Support Vector regression are used for prediction and combined with a genetic algorithm to then impute missing variables. The use of PCA improves the overall performance of the autoencoder network while the use of support vector regression shows promising potential for future investigation. Accuracies of up to 97.4 % on imputation of some of the variables were achieved.

Motivation & Objective

• To evaluate and compare the effectiveness of autoencoders, principal component analysis (PCA), and support vector regression (SVR) in imputing missing data values.
• To investigate how combining these models with a genetic algorithm improves imputation accuracy.
• To assess the individual and combined performance of PCA, autoencoders, and SVR in handling incomplete datasets.
• To identify the most effective configuration for data imputation in real-world applications with missing observations.
• To explore the potential of SVR as a promising method for future data imputation research.

Proposed method

• Utilizes autoencoder neural networks to learn low-dimensional representations of input data for imputation.
• Applies principal component analysis (PCA) to reduce dimensionality and improve the performance of the autoencoder.
• Employs support vector regression (SVR) as a predictive model to estimate missing values based on observed features.
• Integrates a genetic algorithm to optimize hyperparameters across the three models for improved imputation accuracy.
• Combines the outputs of the three models through a hybrid framework to enhance robustness and predictive performance.
• Uses accuracy metrics to evaluate imputation performance across different variables and datasets.

Experimental results

Research questions

• RQ1 How do autoencoders, PCA, and SVR compare in terms of imputation accuracy for datasets with missing values?
• RQ2 To what extent does applying PCA improve the performance of autoencoder-based imputation models?
• RQ3 What is the potential of SVR as a standalone or integrated method for data imputation compared to other models?
• RQ4 Can the integration of multiple models (autoencoder, PCA, SVR) with a genetic algorithm yield superior imputation results?
• RQ5 What level of accuracy can be achieved in imputing missing variables using this hybrid approach?

Key findings

• The hybrid model combining autoencoders, PCA, and SVR with genetic algorithm optimization achieved imputation accuracy of up to 97.4% on certain variables.
• Incorporating PCA significantly improved the overall performance of the autoencoder neural network in the imputation task.
• Support vector regression demonstrated strong potential for future data imputation research, though not the top performer in this study.
• The genetic algorithm effectively optimized model parameters, enhancing the robustness and accuracy of the imputation framework.
• The integration of PCA with autoencoders led to better generalization and reduced reconstruction error in the imputation process.
• The study confirms that model combination with optimization techniques can significantly outperform individual models in missing data recovery.