[Paper Review] Rough extreme learning machine: a new classification method based on uncertainty measure

Lin Feng, Shuliang Xu+2 more|arXiv (Cornell University)|Oct 30, 2017

Machine Learning and ELM41 references3 citations

TL;DR

This paper proposes the Rough Extreme Learning Machine (RELM), a novel classification method that integrates rough set theory with extreme learning machine (ELM) to enhance accuracy and handle high-dimensional data. RELM divides data into upper and lower approximation sets using rough sets, trains separate neurons on each set, and applies attribute reduction to eliminate redundant features, achieving superior accuracy and stability compared to baseline ELM and related models while maintaining fast training speed.

ABSTRACT

Extreme learning machine (ELM) is a new single hidden layer feedback neural network. The weights of the input layer and the biases of neurons in hidden layer are randomly generated, the weights of the output layer can be analytically determined. ELM has been achieved good results for a large number of classification tasks. In this paper, a new extreme learning machine called rough extreme learning machine (RELM) was proposed. RELM uses rough set to divide data into upper approximation set and lower approximation set, and the two approximation sets are utilized to train upper approximation neurons and lower approximation neurons. In addition, an attribute reduction is executed in this algorithm to remove redundant attributes. The experimental results showed, comparing with the comparison algorithms, RELM can get a better accuracy and repeatability in most cases, RELM can not only maintain the advantages of fast speed, but also effectively cope with the classification task for high-dimensional data.

Motivation & Objective

• To address the limitations of traditional ELM in handling noisy and high-dimensional data by incorporating uncertainty modeling.
• To improve classification accuracy and generalization by leveraging rough set theory for data approximation and attribute reduction.
• To maintain the fast learning speed of ELM while enhancing robustness against redundant or irrelevant features.
• To evaluate the performance of RELM across diverse datasets and compare it with existing ELM variants and baseline methods.

Proposed method

• RELM uses rough set theory to partition the training data into upper and lower approximation sets based on decision rules.
• Separate hidden-layer neurons—upper approximation neurons and lower approximation neurons—are trained on the respective approximation sets.
• The output weights are analytically determined using the Moore-Penrose pseudoinverse, preserving the fast learning property of ELM.
• Attribute reduction is applied to remove redundant features, improving model efficiency and reducing overfitting.
• The final classification decision is made by combining the outputs of upper and lower approximation neurons.
• A comparison is made with standard ELM and a variant called URELM (Uncertainty-based RELM) to evaluate performance.

Fig. 1: Rough set diagrammatic sketch

Experimental results

Research questions

• RQ1 Can integrating rough set theory into ELM improve classification accuracy and robustness on high-dimensional and noisy datasets?
• RQ2 How does RELM perform in comparison to standard ELM and other ELM variants in terms of accuracy and stability?
• RQ3 To what extent does attribute reduction in RELM reduce dimensionality without degrading classification performance?
• RQ4 How sensitive is RELM’s time complexity to increasing data dimensions compared to traditional ELM and RS+BP?

Key findings

• RELM achieved higher average accuracy than URELM and standard ELM across 14 benchmark datasets, with a mean accuracy of 0.7563 ± 0.0233 on the adult dataset and 0.7885 ± 0.0395 on hepatitis.
• On the STAGGER dataset, RELM achieved perfect accuracy (1.0000 ± 0.0000), outperforming URELM (0.9886 ± 0.0302).
• Attribute reduction significantly reduced feature dimensions—e.g., from 26 to 3 in the Horse dataset (88.46% reduction) and from 24 to 7 in Germany (70.83% reduction).
• Time overhead of RELM increased with data dimension but remained substantially lower than RS+BP, indicating better scalability and lower sensitivity to dimensionality.
• On the yeast and pima datasets, RELM maintained stable performance (0.6623 ± 0.1245 and 0.5659 ± 0.0974, respectively), while URELM showed lower accuracy and higher variance.
• Despite high performance, RELM showed higher variance in accuracy on some datasets (e.g., 0.1625 on Horse), indicating potential instability in certain conditions.

Fig. 2: The structure of RELM