[Paper Review] Online Semi-Supervised Concept Drift Detection with Density Estimation

Chang Tan, Vincent C. S. Lee+1 more|arXiv (Cornell University)|Sep 25, 2019

Data Stream Mining Techniques25 references4 citations

TL;DR

This paper proposes an online semi-supervised framework for concept drift detection in streaming data using posterior probability density estimation, enabling real-time detection of both real and virtual drift with limited labeled data. The method achieves state-of-the-art prediction performance while maintaining robust drift detection in partially labeled environments.

ABSTRACT

Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(y|X) while the latter is the change in distribution of X without affecting the posterior probabilities. Many approaches on concept drift detection either assume full availability of data labels, y or handle only the virtual drift. In a streaming environment, the assumption of full availability of data labels, y is questioned. On the other hand, approaches that deal with virtual drift failed to address real drift. Rather than improving the state-of-the-art methods, this paper presents a semi-supervised framework to deal with the challenges above. The objective of the proposed framework is to learn from streaming environment with limited data labels, y and detect real drift concurrently. This paper proposes a novel concept drift detection method utilizing the densities of posterior probabilities in partially labeled streaming environments. Experimental results on both synthetic and realworld datasets show that our proposed semi-supervised framework enables the detection of concept drift in such environment while achieving comparable prediction performance to the state-of-the-art methods.

Motivation & Objective

• Address the challenge of concept drift detection in streaming environments where full label availability is impractical.
• Simultaneously detect both real drift (change in p(y|X)) and virtual drift (change in X distribution without affecting p(y|X)) in partially labeled data.
• Develop a scalable, online learning framework that maintains high predictive performance while detecting drift with limited supervision.
• Overcome limitations of existing methods that either assume full labeling or fail to detect real drift in semi-supervised settings.

Proposed method

• Utilize density estimation of posterior probabilities p(y|X) to detect changes in the joint distribution over time.
• Apply an online learning mechanism that updates the density model incrementally as new data arrives.
• Leverage both labeled and unlabeled data in a semi-supervised manner to estimate the posterior density without requiring full labeling.
• Detect concept drift by monitoring statistical divergence in the estimated posterior density across time windows.
• Use kernel density estimation or similar non-parametric methods to model the evolving posterior distribution in a streaming context.
• Integrate drift detection with model retraining to maintain predictive accuracy under concept drift.

Experimental results

Research questions

• RQ1 Can a semi-supervised framework detect concept drift effectively when only a small fraction of labels are available in a streaming setting?
• RQ2 How does the proposed method perform in detecting both real and virtual drift compared to fully supervised or unsupervised alternatives?
• RQ3 To what extent does the method maintain predictive performance under concept drift with limited labeling?
• RQ4 How sensitive is the drift detection mechanism to changes in the posterior distribution when labels are scarce?

Key findings

• The proposed method achieves comparable prediction performance to state-of-the-art fully supervised methods despite using only partial labels.
• The framework successfully detects both real and virtual concept drift in synthetic and real-world datasets under semi-supervised conditions.
• Density estimation of posterior probabilities enables reliable drift detection even when label availability is low.
• The online nature of the method allows for real-time adaptation and detection without retraining from scratch.