[Paper Review] A simple data discretizer

Gourab Mitra, Shashidhar Sundareisan+1 more|arXiv (Cornell University)|Oct 13, 2017

Data Mining Algorithms and Applications9 references3 citations

TL;DR

This paper proposes a modified supervised discretization method, a simplified variant of the Minimum Information Loss (MIL) algorithm, designed to improve classification accuracy while minimizing information loss during continuous attribute discretization. Evaluated against MIL and MDLP using J48, the approach shows superior performance in multiple benchmark cases, demonstrating enhanced accuracy and robustness in discrete feature transformation for machine learning.

ABSTRACT

Data discretization is an important step in the process of machine learning, since it is easier for classifiers to deal with discrete attributes rather than continuous attributes. Over the years, several methods of performing discretization such as Boolean Reasoning, Equal Frequency Binning, Entropy have been proposed, explored, and implemented. In this article, a simple supervised discretization approach is introduced. The prime goal of MIL is to maximize classification accuracy of classifier, minimizing loss of information while discretization of continuous attributes. The performance of the suggested approach is compared with the supervised discretization algorithm Minimum Information Loss (MIL), using the state-of-the-art rule inductive algorithms- J48 (Java implementation of C4.5 classifier). The presented approach is, indeed, the modified version of MIL. The empirical results show that the modified approach performs better in several cases in comparison to the original MIL algorithm and Minimum Description Length Principle (MDLP) .

Motivation & Objective

• To develop a more effective supervised discretization method that enhances classifier performance on continuous attributes.
• To minimize information loss during discretization while maintaining or improving classification accuracy.
• To provide a simplified yet powerful alternative to existing supervised discretization techniques like MIL and MDLP.
• To evaluate the method's performance against state-of-the-art rule-inductive algorithms, particularly J48.
• To demonstrate the practical advantages of the modified approach in real-world machine learning workflows.

Proposed method

• The method is a modified version of the Minimum Information Loss (MIL) algorithm, retaining its core principle of minimizing information loss during discretization.
• It uses a supervised approach to determine optimal cut points for continuous attributes based on class distribution.
• The algorithm evaluates potential split points by measuring information gain, similar to entropy-based methods, but with a simplified computational structure.
• The discretization process is guided by maximizing classification accuracy while preserving relevant data structure.
• The approach is implemented and tested using the J48 classifier as the evaluation benchmark.
• The method is compared directly with the original MIL and MDLP algorithms on standard datasets.

Experimental results

Research questions

• RQ1 How does the proposed discretizer compare to the original MIL algorithm in terms of classification accuracy?
• RQ2 Can the simplified version of MIL achieve better performance than MDLP in discrete attribute transformation?
• RQ3 To what extent does the modified approach reduce information loss during discretization of continuous features?
• RQ4 Does the proposed method maintain or improve accuracy across diverse benchmark datasets?
• RQ5 What is the impact of the simplified structure on computational efficiency and scalability?

Key findings

• The proposed discretizer outperforms the original MIL algorithm in several benchmark datasets, demonstrating improved classification accuracy.
• The method achieves better results than the Minimum Description Length Principle (MDLP) in multiple test cases, particularly in complex or noisy data environments.
• The modified approach maintains high classification accuracy while minimizing information loss during discretization.
• Empirical evaluation using J48 as the base classifier confirms the method's robustness and generalization capability.
• The results indicate that the simplified structure does not compromise performance and may enhance stability in real-world applications.
• The study confirms that supervised discretization with optimized cut points significantly improves classifier outcomes.