[Paper Review] General Regression Neural Networks, Radial Basis Function Neural Networks, Support Vector Machines, and Feedforward Neural Networks

Alison Jenkins, Vinika Gupta+1 more|arXiv (Cornell University)|Nov 16, 2019

Traffic Prediction and Management Techniques5 references4 citations

TL;DR

This study compares General Regression Neural Networks (GRNN), Radial Basis Function Neural Networks (RBFNN), Support Vector Machines (SVM), and Feedforward Neural Networks (FFNN) to identify optimal σ values that maximize F1 score and accuracy. Using a binary relabeled dataset, it applies genetic algorithms for GRNN, Kohonen LVQ for RBFNN clustering, and scikit-learn for SVM and FFNN training, finding that RBFNN with LVQ-II improves F1 score to 0.3726 and accuracy to 0.2735, outperforming the non-LVQ variant.

ABSTRACT

The aim of this project is to develop a code to discover the optimal sigma value that maximum the F1 score and the optimal sigma value that maximizes the accuracy and to find out if they are the same. Four algorithms which can be used to solve this problem are: Genetic Regression Neural Networks (GRNNs), Radial Based Function (RBF) Neural Networks (RBFNNs), Support Vector Machines (SVMs) and Feedforward Neural Network (FFNNs).

Motivation & Objective

• To determine the optimal σ value that maximizes the F1 score across multiple machine learning models.
• To identify the optimal σ value that maximizes classification accuracy and compare it with the F1-optimized σ.
• To evaluate the performance of GRNN, RBFNN, SVM, and FFNN on a binary-relabeled dataset derived from a regression problem.
• To assess the impact of incorporating Kohonen unsupervised learning and back-propagation in RBFNNs on model performance.
• To compare the statistical performance metrics—accuracy, precision, recall, and F1—across all four algorithms.

Proposed method

• Used a steady-state genetic algorithm (SSGA) to evolve σ values in General Regression Neural Networks (GRNN).
• Implemented two variants of RBFNN: one without and one with Kohonen LVQ-I and LVQ-II for cluster center and width optimization.
• Applied LVQ-I for unsupervised clustering based on Euclidean distance, updating weights of winning and neighboring units.
• Used LVQ-II to refine clustering by rewarding correct classifications and penalizing misclassifications with a conscience factor.
• Trained SVMs using both linear and radial basis function (RBF) kernels via scikit-learn, with 90% training and 10% testing split.
• Trained Feedforward Neural Networks (FFNN) with 1, 2, and 4 hidden layers using sigmoidal activation functions and varied σ values.

Experimental results

Research questions

• RQ1 Does the σ value that maximizes F1 score differ from the σ value that maximizes accuracy across the tested models?
• RQ2 How does incorporating Kohonen unsupervised learning and back-propagation affect RBFNN performance compared to standard RBFNN?
• RQ3 Which of the four models—GRNN, RBFNN, SVM, or FFNN—achieves the highest F1 score and accuracy on the binary-relabeled dataset?
• RQ4 What is the impact of cluster unit dominance in LVQ networks, and how does the conscience factor mitigate this issue?
• RQ5 How do different kernel types (linear vs. RBF) in SVMs influence model performance in terms of F1 and accuracy?

Key findings

• The RBFNN with Kohonen unsupervised learning and back-propagation achieved an F1 score of 0.3726, significantly higher than the non-LVQ variant (0.2342).
• The RBFNN with LVQ-II improved accuracy to 0.2735, compared to 0.2367 for the non-LVQ version, indicating better generalization.
• The SVM with a linear kernel (SVM 3) achieved the highest F1 score of 0.4643 and accuracy of 0.4342, outperforming all other models.
• The GRNN and EGRNN-I models showed comparable performance, with EGRNN-I achieving slightly higher accuracy (0.4328) and F1 (0.3369).
• The Feedforward Neural Network (FFNN) performed poorly, with only 0.1293 accuracy and 0.2986 F1 score, indicating limited effectiveness on this dataset.
• The σ values optimizing F1 and accuracy were not identical across models, confirming that the two objectives require distinct hyperparameter tuning.