[Paper Review] Use of Ensembles of Fourier Spectra in Capturing Recurrent Concepts in Data Streams

Sripirakas Sakthithasan, Russel Pears+2 more|arXiv (Cornell University)|Apr 23, 2015

Data Stream Mining Techniques8 references6 citations

TL;DR

This paper proposes an ensemble of Fourier spectra (EP) to efficiently capture and re-use recurring concepts in data streams, outperforming single-spectrum methods in accuracy, memory efficiency, and speed. By aggregating multiple Fourier-encoded decision trees into a unified spectrum with shared coefficients and spectral energy thresholding, EP reduces redundancy and improves robustness in volatile, high-velocity environments.

ABSTRACT

In this research, we apply ensembles of Fourier encoded spectra to capture and mine recurring concepts in a data stream environment. Previous research showed that compact versions of Decision Trees can be obtained by applying the Discrete Fourier Transform to accurately capture recurrent concepts in a data stream. However, in highly volatile environments where new concepts emerge often, the approach of encoding each concept in a separate spectrum is no longer viable due to memory overload and thus in this research we present an ensemble approach that addresses this problem. Our empirical results on real world data and synthetic data exhibiting varying degrees of recurrence reveal that the ensemble approach outperforms the single spectrum approach in terms of classification accuracy, memory and execution time.

Motivation & Objective

• Address the memory overload problem in data stream environments where frequent concept re-occurrence leads to excessive storage of individual Fourier spectra.
• Improve classification accuracy and system stability in volatile data streams by recognizing partial and composite re-occurrences of concepts.
• Reduce computational cost and memory footprint through spectral energy thresholding and optimized Fourier basis computation.
• Enable efficient reuse of previously learned concepts in real-time, safety-critical applications such as autopilot systems and financial forecasting.
• Develop a scalable, robust method for concept re-use that maintains performance under noise and concept drift.

Proposed method

• Aggregate multiple Fourier-encoded decision trees into a single unified spectrum using an ensemble approach to reduce memory overhead.
• Apply spectral energy thresholding to control compression levels by retaining only the most significant Fourier coefficients.
• Optimize DFT computation by eliminating expensive inner product operations through a modified basis function evaluation strategy.
• Use structural similarity between spectra to guide ensemble aggregation, improving recognition of concept re-occurrences and reducing overfitting.
• Integrate the EP method with drift detectors (e.g., SeqDrift2) to trigger spectrum updates at concept drift points.
• Implement parallel processing of spectrum computation and data stream ingestion in a distributed framework like Spark to enhance throughput.

Experimental results

Research questions

• RQ1 Can an ensemble of Fourier spectra outperform single-spectrum encoding in capturing recurring concepts in data streams?
• RQ2 How does spectral energy thresholding affect compression efficiency and classification accuracy in concept re-use?
• RQ3 What impact does the choice of drift detector (e.g., ADWIN vs. SeqDrift2) have on the stability and accuracy of the ensemble method?
• RQ4 How does pool size influence the performance of the ensemble-based approach under varying memory constraints?
• RQ5 Can structural similarity in spectra improve recognition of partially re-occurring or composite concepts compared to accuracy-based aggregation?

Key findings

• EP achieved 7.6% higher classification accuracy than FCT at a pool size of 1, demonstrating superior performance even in extreme memory-constrained environments.
• With a pool size of 10, EP maintained higher accuracy than FCT, showing that ensemble aggregation scales well with available memory.
• Using the SeqDrift2 drift detector reduced accuracy fluctuations by 5% compared to ADWIN, due to fewer false positive detections.
• The ensemble approach with structural similarity outperformed accuracy-based and naive combination methods in robustness to noise and overfitting.
• Spectral energy thresholding enabled effective control over compression levels, directly influencing memory usage and model fidelity.
• The optimized DFT computation eliminated costly inner product operations, improving processing speed without sacrificing accuracy.