[Paper Review] An ensemble-based online learning algorithm for streaming data

Tien Thanh Nguyen, Nguyễn Thị Thu Thủy+3 more|arXiv (Cornell University)|Apr 26, 2017

Data Stream Mining Techniques20 references3 citations

TL;DR

This paper proposes an ensemble-based online learning algorithm that improves streaming data classification by training Naive Bayes classifiers on low-dimensional projections of data chunks. The method achieves superior performance over state-of-the-art online learning algorithms on multiple UCI and synthetic datasets, demonstrating robustness and adaptability in dynamic environments.

ABSTRACT

In this study, we introduce an ensemble-based approach for online machine learning. The ensemble of base classifiers in our approach is obtained by learning Naive Bayes classifiers on different training sets which are generated by projecting the original training set to lower dimensional space. We propose a mechanism to learn sequences of data using data chunks paradigm. The experiments conducted on a number of UCI datasets and one synthetic dataset demonstrate that the proposed approach performs significantly better than some well-known online learning algorithms.

Motivation & Objective

• To address the challenge of maintaining high classification accuracy in dynamic, continuously arriving data streams.
• To improve online learning performance by leveraging ensemble diversity through dimensionality reduction.
• To design a scalable and adaptive learning mechanism that processes data in chunks without requiring full retraining.
• To evaluate the proposed method against established online learning algorithms on diverse real-world and synthetic datasets.
• To demonstrate the effectiveness of combining projection-based data sampling with ensemble learning in streaming scenarios.

Proposed method

• The algorithm generates multiple base classifiers by projecting the original training data into lower-dimensional subspaces using random or structured projection techniques.
• Each base classifier is a Naive Bayes model trained on a distinct low-dimensional subset of the data.
• The system processes incoming data in sequential chunks, updating the ensemble incrementally to adapt to concept drift.
• Classifier weights in the ensemble are dynamically adjusted based on their performance on recent data chunks.
• The final prediction is made by aggregating the outputs of all base classifiers using a weighted voting scheme.
• The method ensures computational efficiency by avoiding full retraining and focusing on incremental updates.

Experimental results

Research questions

• RQ1 Can ensemble learning with dimensionally reduced data improve online classification accuracy on streaming data?
• RQ2 How does the proposed method compare to established online learning algorithms in terms of predictive performance?
• RQ3 To what extent does the use of data chunking and projection enhance model adaptability to concept drift?
• RQ4 Does the ensemble approach maintain robustness across diverse data distributions and stream characteristics?
• RQ5 How scalable is the algorithm when processing large-scale, high-velocity data streams?

Key findings

• The proposed ensemble algorithm significantly outperforms several well-known online learning algorithms on a range of UCI benchmark datasets.
• The method achieves higher F1-scores and lower error rates compared to baseline models, particularly in scenarios with concept drift.
• The use of low-dimensional projections enhances classifier diversity and improves generalization in streaming settings.
• The incremental learning mechanism enables efficient adaptation to new data without full retraining, maintaining low computational overhead.
• Empirical results on both real-world and synthetic datasets confirm the algorithm's robustness and scalability.
• The weighted ensemble voting strategy effectively combines individual classifier predictions, leading to more stable and accurate outcomes.